[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:23 dist-test-slave-1zqn krb5kdc[7597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:23 dist-test-slave-1zqn krb5kdc[7597](info): set up 2 sockets
Aug 09 00:37:23 dist-test-slave-1zqn krb5kdc[7597](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 00:37:24.231418 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:24.371541 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43087
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:24.454880 7613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:24.455137 7613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:24.455178 7613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:24.456368 7613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:24.456462 7613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:24.456501 7613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:24.456526 7613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:24.456588 7613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:24.458125 7613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:24.458352 7613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:24.458549 7613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:24.464223 7621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:24.464226 7613 server_base.cc:1034] running on GCE node
W20260809 00:37:24.464207 7619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:24.464203 7618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:24.464797 7613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:24.465168 7613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:24.466327 7613 hybrid_clock.cc:648] HybridClock initialized: now 1786235844466263 us; error 81 us; skew 500 ppm
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:24.469267 7613 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:24.469802 7613 webserver.cc:460] Webserver started at http://127.7.105.126:42461/ using document root <none> and password file <none>
I20260809 00:37:24.470084 7613 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:24.470160 7613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:24.470332 7613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:24.471488 7613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f6226480370745d988f27e2a5c35eaa6"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "f9f01704cca82b07afde1a83216ac22c"
server_key_iv: "bebc2f335462566d8b3ddc93f1941a8c"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.471863 7613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f6226480370745d988f27e2a5c35eaa6"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "f9f01704cca82b07afde1a83216ac22c"
server_key_iv: "bebc2f335462566d8b3ddc93f1941a8c"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.473413 7613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:24.474324 7628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.474606 7613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:24.474730 7613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f6226480370745d988f27e2a5c35eaa6"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "f9f01704cca82b07afde1a83216ac22c"
server_key_iv: "bebc2f335462566d8b3ddc93f1941a8c"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.474818 7613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:24.495731 7613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:24.498368 7613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:24.498545 7613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:24.505513 7613 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:43087
I20260809 00:37:24.505995 7685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:43087 every 8 connection(s)
I20260809 00:37:24.506063 7613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:24.506860 7686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:24.507500 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 7613
I20260809 00:37:24.507623 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:24.508431 7589 external_mini_cluster.cc:1428] Setting key d3da3d2ee682012d85f430a90b40e806
I20260809 00:37:24.510072 7686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6: Bootstrap starting.
I20260809 00:37:24.511708 7686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:24.512913 7686 log.cc:824] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:24.514046 7686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6: No bootstrap required, opened a new log
I20260809 00:37:24.518821 7686 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } }
I20260809 00:37:24.518971 7686 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:24.518994 7686 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6226480370745d988f27e2a5c35eaa6, State: Initialized, Role: FOLLOWER
I20260809 00:37:24.519130 7686 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } }
I20260809 00:37:24.519234 7686 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:24.519313 7686 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:24.519368 7686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:24.520120 7686 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } }
I20260809 00:37:24.520254 7686 leader_election.cc:302] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6226480370745d988f27e2a5c35eaa6; no voters:
I20260809 00:37:24.520466 7686 leader_election.cc:288] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:24.520537 7693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:24.520819 7693 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [term 1 LEADER]: Becoming Leader. State: Replica: f6226480370745d988f27e2a5c35eaa6, State: Running, Role: LEADER
I20260809 00:37:24.520953 7686 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:24.520969 7693 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } }
I20260809 00:37:24.521670 7695 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6226480370745d988f27e2a5c35eaa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } } }
I20260809 00:37:24.521791 7695 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:24.522032 7696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6226480370745d988f27e2a5c35eaa6. Latest consensus state: current_term: 1 leader_uuid: "f6226480370745d988f27e2a5c35eaa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6226480370745d988f27e2a5c35eaa6" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43087 } } }
I20260809 00:37:24.522101 7696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:24.522769 7699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:24.523926 7699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:24.526849 7699 catalog_manager.cc:1347] Generated new cluster ID: 969c0c471d4947e3b09fa88d7cfda883
I20260809 00:37:24.526904 7699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:24.531507 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:24.509426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58502 (local address 127.7.105.126:43087)
0809 00:37:24.509798 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:24.509805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:24.509833 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:37:24.514613 (+ 4780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:24.514620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:24.514689 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:24.514811 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:24.518007 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.519318 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.520610 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.521305 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.521431 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:24.527023 (+ 5592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:24.527028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:24.527028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:24.527043 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:24.529334 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.529802 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.529808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.529808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.529898 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.530181 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.530182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.530183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.530426 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:24.530748 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:24.530960 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 00:37:24.531108 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":188,"threads_started":1}
I20260809 00:37:24.555184 7699 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:24.556079 7699 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:24.561343 7699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6226480370745d988f27e2a5c35eaa6: Generated new TSK 0
I20260809 00:37:24.561542 7699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:24.599637 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:24.695415 7720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:24.695638 7720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:24.695663 7720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:24.696794 7720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:24.696830 7720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:24.696856 7720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:24.698225 7720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:24.698408 7720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:24.698594 7720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:24.704249 7729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:24.704320 7726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:24.704282 7727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:24.704314 7720 server_base.cc:1034] running on GCE node
I20260809 00:37:24.704684 7720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:24.704941 7720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:24.706115 7720 hybrid_clock.cc:648] HybridClock initialized: now 1786235844706103 us; error 36 us; skew 500 ppm
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:24.708834 7720 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:24.709214 7720 webserver.cc:460] Webserver started at http://127.7.105.65:37695/ using document root <none> and password file <none>
I20260809 00:37:24.709398 7720 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:24.709437 7720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:24.709546 7720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:24.710562 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "da4a006c4bf541c8987086ba407bb5e3"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "a59cb7bfb35df70004f3a2c87a840c37"
server_key_iv: "7a5b502bb4143a7bf4f6da4cc328afd9"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.710870 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "da4a006c4bf541c8987086ba407bb5e3"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "a59cb7bfb35df70004f3a2c87a840c37"
server_key_iv: "7a5b502bb4143a7bf4f6da4cc328afd9"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.712316 7720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:24.713042 7739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.713244 7720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:24.713331 7720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "da4a006c4bf541c8987086ba407bb5e3"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "a59cb7bfb35df70004f3a2c87a840c37"
server_key_iv: "7a5b502bb4143a7bf4f6da4cc328afd9"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.713387 7720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:24.724159 7720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:24.726176 7720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:24.726292 7720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:24.726504 7720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:24.726840 7720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:24.726873 7720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.726927 7720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:24.726953 7720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.740474 7720 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:36647
I20260809 00:37:24.740672 7870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:36647 every 8 connection(s)
I20260809 00:37:24.740968 7720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:24.745653 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 7720
I20260809 00:37:24.745735 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:24.745965 7589 external_mini_cluster.cc:1428] Setting key 8fb69d959977dd2a2ed988e250ae261d
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:24.752652 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:24.741979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53993 (local address 127.7.105.126:43087)
0809 00:37:24.742047 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:24.742049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:24.742325 (+ 276us) server_negotiation.cc:407] Connection header received
0809 00:37:24.743177 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:24.743178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:24.743189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:24.743257 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:24.744544 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.745073 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.746320 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.746573 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.746639 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:24.749672 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:24.749674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:24.749674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:24.749682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:24.751194 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.751610 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.751611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.751611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.751672 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.751936 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.751937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.751937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.752090 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:24.752157 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:24.752461 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 00:37:24.752559 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:37:24.752931 7874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:24.741941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43087 (local address 127.7.105.65:53993)
0809 00:37:24.742177 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:24.742217 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:24.743060 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:24.743333 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:24.743336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:24.744142 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:24.744447 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:24.744452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.745161 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.745161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:24.746245 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:24.746248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.746663 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.746664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:24.746854 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:24.747517 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:24.747532 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:24.749545 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:24.751287 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:24.751295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:24.751298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:24.751529 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:24.751744 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:24.751745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:24.751745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:24.751860 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:24.752148 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:24.752156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:24.752301 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 00:37:24.752409 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:37:24.752578 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260809 00:37:24.753769 7872 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43087
I20260809 00:37:24.753885 7872 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:24.754079 7872 heartbeater.cc:511] Master 127.7.105.126:43087 requested a full tablet report, sending...
I20260809 00:37:24.754689 7647 ts_manager.cc:194] Registered new tserver with Master: da4a006c4bf541c8987086ba407bb5e3 (127.7.105.65:36647)
I20260809 00:37:24.755347 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:53993
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:24.779511 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:24.873924 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:24.874194 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:24.874235 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:24.875411 7877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:24.875486 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:24.875535 7877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:24.877141 7877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:24.877419 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:24.877669 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:24.883687 7885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:24.883807 7882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:24.883831 7883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:24.883882 7877 server_base.cc:1034] running on GCE node
I20260809 00:37:24.884191 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:24.884404 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:24.885574 7877 hybrid_clock.cc:648] HybridClock initialized: now 1786235844885556 us; error 33 us; skew 500 ppm
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:24.888159 7877 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:24.888654 7877 webserver.cc:460] Webserver started at http://127.7.105.66:43279/ using document root <none> and password file <none>
I20260809 00:37:24.888890 7877 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:24.888959 7877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:24.889124 7877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:24.890241 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "3fd8e09918954f998f79a80551f86007"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "7e163de60575b57548ab14b22a74aba9"
server_key_iv: "ea023b47c11478fc69798e90a1527763"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.890642 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "3fd8e09918954f998f79a80551f86007"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "7e163de60575b57548ab14b22a74aba9"
server_key_iv: "ea023b47c11478fc69798e90a1527763"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.892244 7877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:24.893121 7894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.893325 7877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:24.893428 7877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "3fd8e09918954f998f79a80551f86007"
format_stamp: "Formatted at 2026-08-09 00:37:24 on dist-test-slave-1zqn"
server_key: "7e163de60575b57548ab14b22a74aba9"
server_key_iv: "ea023b47c11478fc69798e90a1527763"
server_key_version: "encryptionkey@0"
I20260809 00:37:24.893517 7877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:24.915241 7877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:24.917348 7877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:24.917515 7877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:24.917764 7877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:24.918102 7877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:24.918159 7877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.918198 7877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:24.918227 7877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:24.926115 7877 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:44913
I20260809 00:37:24.926222 8022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:44913 every 8 connection(s)
I20260809 00:37:24.926542 7877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:37:24 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235844, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:24.935789 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:24.927326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37143 (local address 127.7.105.126:43087)
0809 00:37:24.927390 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:24.927392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:24.927658 (+ 266us) server_negotiation.cc:407] Connection header received
0809 00:37:24.928267 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:24.928268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:24.928281 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:24.928338 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:24.929310 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.929810 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.930800 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.931000 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.931099 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:24.933187 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:24.933189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:24.933189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:24.933196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:24.934640 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.935008 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.935009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.935009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.935060 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:24.935266 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:24.935266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:24.935267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:24.935367 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:24.935416 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:24.935656 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 00:37:24.935722 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:37:24.936000 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:24.927294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43087 (local address 127.7.105.66:37143)
0809 00:37:24.927493 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:24.927531 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:24.928161 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:24.928416 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:24.928419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:24.929003 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:24.929198 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:24.929203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.929907 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.929908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:24.930707 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:24.930709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:24.931109 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:24.931110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:24.931234 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:24.931705 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:24.931713 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:24.933097 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:24.934722 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:24.934728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:24.934730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:24.934944 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:24.935118 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:24.935118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:24.935119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:24.935213 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:24.935420 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:24.935424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:24.935538 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 00:37:24.935631 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:37:24.935749 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260809 00:37:24.936180 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 7877
I20260809 00:37:24.936283 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:24.936489 7589 external_mini_cluster.cc:1428] Setting key 543c17cc2f5f9f5f62813e98005e8183
I20260809 00:37:24.936691 8023 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43087
I20260809 00:37:24.936797 8023 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:24.936977 8023 heartbeater.cc:511] Master 127.7.105.126:43087 requested a full tablet report, sending...
I20260809 00:37:24.937374 7647 ts_manager.cc:194] Registered new tserver with Master: 3fd8e09918954f998f79a80551f86007 (127.7.105.66:44913)
I20260809 00:37:24.937870 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:37143
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:24.963210 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:25.048239 8029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:25.048640 8029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:25.048705 8029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:25.049909 8029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:25.049961 8029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:25.049989 8029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:25.051493 8029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:25.051756 8029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:25.052023 8029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:25.057313 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:25.057337 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:25.057337 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:25.057652 8029 server_base.cc:1034] running on GCE node
I20260809 00:37:25.057924 8029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:25.058179 8029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:25.059336 8029 hybrid_clock.cc:648] HybridClock initialized: now 1786235845059322 us; error 30 us; skew 500 ppm
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:25.062278 8029 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:25.062731 8029 webserver.cc:460] Webserver started at http://127.7.105.67:39963/ using document root <none> and password file <none>
I20260809 00:37:25.062985 8029 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:25.063041 8029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:25.063174 8029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:25.064322 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "8608ecfc3b98433bbe9352ee2c81cea3"
format_stamp: "Formatted at 2026-08-09 00:37:25 on dist-test-slave-1zqn"
server_key: "af635f9bd7c77c7ebae466b3101d8c29"
server_key_iv: "b4a29027129b8e79d2b58b2285ca9f5d"
server_key_version: "encryptionkey@0"
I20260809 00:37:25.064687 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "8608ecfc3b98433bbe9352ee2c81cea3"
format_stamp: "Formatted at 2026-08-09 00:37:25 on dist-test-slave-1zqn"
server_key: "af635f9bd7c77c7ebae466b3101d8c29"
server_key_iv: "b4a29027129b8e79d2b58b2285ca9f5d"
server_key_version: "encryptionkey@0"
I20260809 00:37:25.066113 8029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:25.066843 8045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:25.067010 8029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:25.067119 8029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "8608ecfc3b98433bbe9352ee2c81cea3"
format_stamp: "Formatted at 2026-08-09 00:37:25 on dist-test-slave-1zqn"
server_key: "af635f9bd7c77c7ebae466b3101d8c29"
server_key_iv: "b4a29027129b8e79d2b58b2285ca9f5d"
server_key_version: "encryptionkey@0"
I20260809 00:37:25.067198 8029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:25.080230 8029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:25.082213 8029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:25.082404 8029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:25.082674 8029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:25.083011 8029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:25.083068 8029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:25.083104 8029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:25.083133 8029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:25.090467 8029 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:45687
I20260809 00:37:25.090512 8169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:45687 every 8 connection(s)
I20260809 00:37:25.090898 8029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:25.098992 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8029
I20260809 00:37:25.099094 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:25.099263 7589 external_mini_cluster.cc:1428] Setting key 854975b1fded565490ce4c993a37a603
I20260809 00:37:25.101143 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:25.091714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:46951 (local address 127.7.105.126:43087)
0809 00:37:25.091790 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:25.091792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:25.092030 (+ 238us) server_negotiation.cc:407] Connection header received
0809 00:37:25.092627 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:25.092629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:25.092642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:25.092698 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:25.093709 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.094218 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.095255 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.095442 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.095535 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:25.097859 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:25.097860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:25.097860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:25.097868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:25.099598 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.100109 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.100110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.100111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.100177 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.100449 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.100450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.100451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.100576 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:25.100651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:25.100943 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 00:37:25.101051 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:37:25.101316 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:25.091702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43087 (local address 127.7.105.67:46951)
0809 00:37:25.091900 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:25.091938 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:25.092506 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:25.092785 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:25.092791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:25.093398 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:25.093602 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:25.093607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.094322 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.094323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:25.095145 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:25.095147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.095538 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.095538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:25.095680 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:25.096210 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:25.096219 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:25.097763 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:25.099679 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:25.099687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:25.099689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:25.099993 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:25.100259 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:25.100260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:25.100260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:25.100373 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:25.100654 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:25.100661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:25.100804 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:37:25.100907 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:37:25.101072 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260809 00:37:25.101822 8170 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43087
I20260809 00:37:25.101917 8170 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:25.102072 8170 heartbeater.cc:511] Master 127.7.105.126:43087 requested a full tablet report, sending...
I20260809 00:37:25.102397 7647 ts_manager.cc:194] Registered new tserver with Master: 8608ecfc3b98433bbe9352ee2c81cea3 (127.7.105.67:45687)
I20260809 00:37:25.102773 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:46951
I20260809 00:37:25.111804 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:25.137809 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:25.129693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.7.105.65:36647)
0809 00:37:25.129962 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:25.129968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:25.129979 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:25.130008 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:25.130009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:25.130029 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:25.130098 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:25.130878 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.131633 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.132457 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.132773 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.132825 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:25.135243 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:25.135250 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:25.135250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:25.135256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:25.136763 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.137139 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.137142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.137142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.137196 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.137375 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.137376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.137376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.137504 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:25.137628 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:25.137666 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:37:25.137730 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":188,"threads_started":1}
W20260809 00:37:25.138262 7774 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52152
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[7597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:25.162858 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:25.155720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.7.105.65:36647)
0809 00:37:25.155874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:25.155877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:25.155888 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:25.156082 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:25.156083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:25.156096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:25.156150 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:25.156786 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.157284 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.158071 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:25.158280 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:25.158357 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:25.160644 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:25.160645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:25.160645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:25.160652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:25.161930 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.162245 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.162246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.162246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.162300 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:25.162464 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:25.162465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:25.162465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:25.162563 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:25.162639 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:25.162724 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 00:37:25.162789 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260809 00:37:25.163461 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 7720
I20260809 00:37:25.173193 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 7877
I20260809 00:37:25.180164 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8029
I20260809 00:37:25.186463 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 7613
2026-08-09T00:37:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2157 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[8185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[8185](info): set up 2 sockets
Aug 09 00:37:25 dist-test-slave-1zqn krb5kdc[8185](info): commencing operation
krb5kdc: starting...
W20260809 00:37:26.362051 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:26.491015 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41993
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:26.573933 8203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:26.574167 8203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:26.574190 8203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:26.575289 8203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:26.575326 8203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:26.575340 8203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:26.575354 8203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:26.575366 8203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:26.576810 8203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:26.577016 8203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:26.577212 8203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:26.582406 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:26.582460 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.582664 8203 server_base.cc:1034] running on GCE node
W20260809 00:37:26.582404 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.582983 8203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:26.583302 8203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:26.584489 8203 hybrid_clock.cc:648] HybridClock initialized: now 1786235846584472 us; error 49 us; skew 500 ppm
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:26.587245 8203 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:26.587795 8203 webserver.cc:460] Webserver started at http://127.7.105.126:35251/ using document root <none> and password file <none>
I20260809 00:37:26.588039 8203 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:26.588164 8203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:26.588327 8203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:26.589501 8203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "bebb092ce1de44aa8db5abc6efd09dae"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "ad46af76e2a3720a5fff1c930c82cf71"
server_key_iv: "4325506db2d62e72692fb2804a080f7f"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.589874 8203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "bebb092ce1de44aa8db5abc6efd09dae"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "ad46af76e2a3720a5fff1c930c82cf71"
server_key_iv: "4325506db2d62e72692fb2804a080f7f"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.591295 8203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:26.592147 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.592397 8203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:26.592535 8203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "bebb092ce1de44aa8db5abc6efd09dae"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "ad46af76e2a3720a5fff1c930c82cf71"
server_key_iv: "4325506db2d62e72692fb2804a080f7f"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.592613 8203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:26.607904 8203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:26.610554 8203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:26.610738 8203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:26.615119 8203 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:41993
I20260809 00:37:26.615154 8274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:41993 every 8 connection(s)
I20260809 00:37:26.615546 8203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:26.616595 8275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:26.616655 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8203
I20260809 00:37:26.616740 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:26.616920 7589 external_mini_cluster.cc:1428] Setting key 876c855cc889582075d536b926a8e55b
I20260809 00:37:26.620040 8275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae: Bootstrap starting.
I20260809 00:37:26.620978 8275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:26.621318 8275 log.cc:824] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:26.622150 8275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae: No bootstrap required, opened a new log
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:26.624886 8275 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } }
I20260809 00:37:26.625053 8275 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:26.625106 8275 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebb092ce1de44aa8db5abc6efd09dae, State: Initialized, Role: FOLLOWER
I20260809 00:37:26.625249 8275 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } }
I20260809 00:37:26.625352 8275 raft_consensus.cc:388] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:26.625399 8275 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:26.625449 8275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:26.626000 8275 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } }
I20260809 00:37:26.626124 8275 leader_election.cc:302] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bebb092ce1de44aa8db5abc6efd09dae; no voters:
I20260809 00:37:26.626318 8275 leader_election.cc:288] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:26.626379 8281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:26.626606 8275 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:26.626629 8281 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [term 1 LEADER]: Becoming Leader. State: Replica: bebb092ce1de44aa8db5abc6efd09dae, State: Running, Role: LEADER
I20260809 00:37:26.626847 8281 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } }
I20260809 00:37:26.627254 8282 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bebb092ce1de44aa8db5abc6efd09dae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } } }
I20260809 00:37:26.627347 8282 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:26.627808 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebb092ce1de44aa8db5abc6efd09dae. Latest consensus state: current_term: 1 leader_uuid: "bebb092ce1de44aa8db5abc6efd09dae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebb092ce1de44aa8db5abc6efd09dae" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41993 } } }
I20260809 00:37:26.627964 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:26.628171 8291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:26.628854 8291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:26.629077 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:26.617509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.7.105.126:41993)
0809 00:37:26.617689 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:26.617697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:26.617722 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:37:26.618217 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:26.618223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:26.618272 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:26.618386 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:26.619243 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.620235 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.621343 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.621820 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.621899 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:26.624697 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:26.624701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:26.624701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:26.624712 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:26.626905 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.627403 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.627407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.627407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.627548 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.627791 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.627792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.627792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.628185 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:26.628395 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:26.628622 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 00:37:26.628767 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260809 00:37:26.630483 8291 catalog_manager.cc:1347] Generated new cluster ID: 0e611a783b0c498496f60b49e16a3d88
I20260809 00:37:26.630569 8291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:26.644235 8291 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:26.644982 8291 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:26.650918 8291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bebb092ce1de44aa8db5abc6efd09dae: Generated new TSK 0
I20260809 00:37:26.651140 8291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:26.688414 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:26.774115 8306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:26.774372 8306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:26.774402 8306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:26.775574 8306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:26.775614 8306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:26.775640 8306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:26.777118 8306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:26.777324 8306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:26.777524 8306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:26.783075 8313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:26.783084 8312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.783262 8306 server_base.cc:1034] running on GCE node
W20260809 00:37:26.783087 8316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.783528 8306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:26.783790 8306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:26.785012 8306 hybrid_clock.cc:648] HybridClock initialized: now 1786235846784994 us; error 28 us; skew 500 ppm
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:26.787561 8306 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:26.788127 8306 webserver.cc:460] Webserver started at http://127.7.105.65:35697/ using document root <none> and password file <none>
I20260809 00:37:26.788357 8306 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:26.788426 8306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:26.788575 8306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:26.789635 8306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "fea4b6cf21394fffb740c497088df3fb"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "30752ebb2d38cdb428435d67b0abbaa8"
server_key_iv: "ca1aea979a48d18fad73a319592a364e"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.790017 8306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "fea4b6cf21394fffb740c497088df3fb"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "30752ebb2d38cdb428435d67b0abbaa8"
server_key_iv: "ca1aea979a48d18fad73a319592a364e"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.791360 8306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:26.792153 8323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.792332 8306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:26.792438 8306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "fea4b6cf21394fffb740c497088df3fb"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "30752ebb2d38cdb428435d67b0abbaa8"
server_key_iv: "ca1aea979a48d18fad73a319592a364e"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.792519 8306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:26.818377 8306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:26.820545 8306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:26.820724 8306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:26.821002 8306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:26.821336 8306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:26.821398 8306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.821439 8306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:26.821472 8306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.827648 8306 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:45057
I20260809 00:37:26.827759 8447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:45057 every 8 connection(s)
I20260809 00:37:26.828107 8306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:26.834728 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8306
I20260809 00:37:26.834802 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:26.835016 7589 external_mini_cluster.cc:1428] Setting key 1a5f04910712e79e0269774d9a819082
I20260809 00:37:26.838200 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:26.828954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:56407 (local address 127.7.105.126:41993)
0809 00:37:26.829005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:26.829007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:26.829216 (+ 209us) server_negotiation.cc:407] Connection header received
0809 00:37:26.830282 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:26.830283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:26.830293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:26.830352 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:26.830932 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.831437 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.832570 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.832769 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.832820 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:26.834993 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:26.834995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:26.834995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:26.835005 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:26.836749 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.837175 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.837177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.837177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.837240 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.837478 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.837479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.837480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.837634 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:26.837708 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:26.838063 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 00:37:26.838136 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:26.838521 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:26.828902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41993 (local address 127.7.105.65:56407)
0809 00:37:26.829081 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:26.829126 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:26.830185 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:26.830407 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:26.830410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:26.830663 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:26.830854 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:26.830858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.831588 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.831589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:26.832483 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:26.832486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.832846 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.832847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:26.832947 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:26.833419 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:26.833427 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:26.834884 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:26.836844 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:26.836851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:26.836853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:26.837103 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:26.837308 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:26.837308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:26.837309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:26.837411 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:26.837712 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:26.837720 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:26.837896 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 00:37:26.838024 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 00:37:26.838253 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260809 00:37:26.839079 8448 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41993
I20260809 00:37:26.839190 8448 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:26.839376 8448 heartbeater.cc:511] Master 127.7.105.126:41993 requested a full tablet report, sending...
I20260809 00:37:26.839956 8238 ts_manager.cc:194] Registered new tserver with Master: fea4b6cf21394fffb740c497088df3fb (127.7.105.65:45057)
I20260809 00:37:26.840607 8238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:56407
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:26.863772 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:26.947585 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:26.947783 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:26.947806 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:26.948999 8454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:26.949045 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:26.949072 8454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:26.950570 8454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:26.950781 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:26.950994 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:26.956229 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:26.956248 8460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.956254 8454 server_base.cc:1034] running on GCE node
W20260809 00:37:26.956229 8459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:26.956748 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:26.956985 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:26.958150 8454 hybrid_clock.cc:648] HybridClock initialized: now 1786235846958136 us; error 28 us; skew 500 ppm
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:26.960783 8454 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:26.961258 8454 webserver.cc:460] Webserver started at http://127.7.105.66:39889/ using document root <none> and password file <none>
I20260809 00:37:26.961485 8454 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:26.961560 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:26.961709 8454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:26.962743 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "7cae73f75d5c40549a4fc1070c9bc4f7"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "0c2ac8f41ffd27cfac641fd4c0d8ea08"
server_key_iv: "f2dd22800d59dd55e36ae053aba858a5"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.963102 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "7cae73f75d5c40549a4fc1070c9bc4f7"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "0c2ac8f41ffd27cfac641fd4c0d8ea08"
server_key_iv: "f2dd22800d59dd55e36ae053aba858a5"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.964519 8454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:26.965426 8471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.965698 8454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:26.965821 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "7cae73f75d5c40549a4fc1070c9bc4f7"
format_stamp: "Formatted at 2026-08-09 00:37:26 on dist-test-slave-1zqn"
server_key: "0c2ac8f41ffd27cfac641fd4c0d8ea08"
server_key_iv: "f2dd22800d59dd55e36ae053aba858a5"
server_key_version: "encryptionkey@0"
I20260809 00:37:26.965909 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:26.980026 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:26.982097 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:26.982267 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:26.982518 8454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:26.982841 8454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:26.982895 8454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.982950 8454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:26.982976 8454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:26.989332 8454 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:44743
I20260809 00:37:26.989382 8596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:44743 every 8 connection(s)
I20260809 00:37:26.989734 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:37:26 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:26.999135 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8454
I20260809 00:37:26.999229 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:26.999411 7589 external_mini_cluster.cc:1428] Setting key 2600e2de35d70de5864e35feeaf2c022
I20260809 00:37:27.000150 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:26.990543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:59033 (local address 127.7.105.126:41993)
0809 00:37:26.990593 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:26.990595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:26.990842 (+ 247us) server_negotiation.cc:407] Connection header received
0809 00:37:26.991850 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:26.991852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:26.991865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:26.991923 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:26.992680 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.993192 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.994214 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.994414 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.994508 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:26.997177 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:26.997178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:26.997178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:26.997185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:26.998503 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.998945 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.998946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.998946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.999000 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:26.999359 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:26.999360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:26.999361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:26.999479 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:26.999580 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:26.999910 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 00:37:26.999994 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:27.000322 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:26.990522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41993 (local address 127.7.105.66:59033)
0809 00:37:26.990713 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:26.990751 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:26.991729 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:26.992002 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:26.992007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:26.992374 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:26.992580 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:26.992584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.993292 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.993293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:26.994119 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:26.994121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:26.994521 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:26.994522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:26.994696 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:26.995439 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:26.995448 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:26.997090 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:26.998589 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:26.998596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:26.998598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:26.998829 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:26.999072 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:26.999073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:26.999074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:26.999226 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:26.999588 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:26.999595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:26.999755 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 00:37:26.999867 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:37:26.999992 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260809 00:37:27.000816 8597 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41993
I20260809 00:37:27.000932 8597 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:27.001142 8597 heartbeater.cc:511] Master 127.7.105.126:41993 requested a full tablet report, sending...
I20260809 00:37:27.001536 8238 ts_manager.cc:194] Registered new tserver with Master: 7cae73f75d5c40549a4fc1070c9bc4f7 (127.7.105.66:44743)
I20260809 00:37:27.001915 8238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:59033
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:27.027357 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:27.114162 8602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:27.114374 8602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:27.114399 8602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:27.115597 8602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:27.115640 8602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:27.115667 8602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:27.117290 8602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:27.117558 8602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:27.117820 8602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:27.123015 8608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:27.123016 8610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:27.123016 8607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:27.123339 8602 server_base.cc:1034] running on GCE node
I20260809 00:37:27.123590 8602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:27.123843 8602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:27.125025 8602 hybrid_clock.cc:648] HybridClock initialized: now 1786235847124994 us; error 61 us; skew 500 ppm
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:27.128153 8602 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:27.128640 8602 webserver.cc:460] Webserver started at http://127.7.105.67:33075/ using document root <none> and password file <none>
I20260809 00:37:27.128885 8602 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:27.128932 8602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:27.129072 8602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:27.130146 8602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "f9d26b6aafd64cb397106186cc153d8a"
format_stamp: "Formatted at 2026-08-09 00:37:27 on dist-test-slave-1zqn"
server_key: "fb3c2fea35f434913ada2b837032a4eb"
server_key_iv: "e7c941b22b995b19a2bfaca44cbeb65a"
server_key_version: "encryptionkey@0"
I20260809 00:37:27.130511 8602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "f9d26b6aafd64cb397106186cc153d8a"
format_stamp: "Formatted at 2026-08-09 00:37:27 on dist-test-slave-1zqn"
server_key: "fb3c2fea35f434913ada2b837032a4eb"
server_key_iv: "e7c941b22b995b19a2bfaca44cbeb65a"
server_key_version: "encryptionkey@0"
I20260809 00:37:27.131907 8602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:27.132771 8617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:27.132936 8602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:27.133023 8602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "f9d26b6aafd64cb397106186cc153d8a"
format_stamp: "Formatted at 2026-08-09 00:37:27 on dist-test-slave-1zqn"
server_key: "fb3c2fea35f434913ada2b837032a4eb"
server_key_iv: "e7c941b22b995b19a2bfaca44cbeb65a"
server_key_version: "encryptionkey@0"
I20260809 00:37:27.133086 8602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:27.149704 8602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:27.152215 8602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:27.152336 8602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:27.152568 8602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:27.152947 8602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:27.153004 8602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:27.153057 8602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:27.153083 8602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:27.160424 8602 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:39873
I20260809 00:37:27.160467 8739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:39873 every 8 connection(s)
I20260809 00:37:27.160825 8602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:27.163096 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8602
I20260809 00:37:27.163187 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:27.163345 7589 external_mini_cluster.cc:1428] Setting key d11605c01fde1ebb10f001a95a188ec1
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:27.170728 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.161611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36533 (local address 127.7.105.126:41993)
0809 00:37:27.161675 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.161677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.161929 (+ 252us) server_negotiation.cc:407] Connection header received
0809 00:37:27.162916 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.162918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.162933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.163006 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:27.163898 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.164486 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.165480 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.165671 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.165722 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.167852 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:27.167853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:27.167854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:27.167861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:27.169407 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.169827 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.169828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.169828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.169889 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.170117 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.170117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.170118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.170226 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:27.170281 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.170589 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.170662 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:37:27.170926 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.161633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41993 (local address 127.7.105.67:36533)
0809 00:37:27.161801 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.161838 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.162801 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.163079 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.163083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.163486 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:27.163804 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.163812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.164594 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.164595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.165404 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.165406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.165746 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.165747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.165856 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.166350 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:27.166359 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:27.167765 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:27.169479 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:27.169485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:27.169487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:27.169730 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:27.169949 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:27.169949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:27.169950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:27.170053 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:27.170319 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:27.170326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:27.170466 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.170570 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.170700 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260809 00:37:27.171432 8740 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41993
I20260809 00:37:27.171535 8740 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:27.171684 8740 heartbeater.cc:511] Master 127.7.105.126:41993 requested a full tablet report, sending...
I20260809 00:37:27.172088 8238 ts_manager.cc:194] Registered new tserver with Master: f9d26b6aafd64cb397106186cc153d8a (127.7.105.67:39873)
I20260809 00:37:27.172479 8238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:36533
I20260809 00:37:27.175127 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:27.183079 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.176900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.7.105.126:41993)
0809 00:37:27.176981 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.176984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.177103 (+ 119us) server_negotiation.cc:407] Connection header received
0809 00:37:27.177183 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.177184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.177194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.177264 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:27.177839 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.178325 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.179103 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.179303 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.179379 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.180646 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:27.180648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:27.180649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:27.180657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:27.181863 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.182225 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.182226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.182226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.182288 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.182496 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.182497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.182497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.182657 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:27.182768 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.182889 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.182971 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:37:27.185413 8238 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:27.186988 8238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:27.193850 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:44743 (local address 127.0.0.1:35990)
0809 00:37:27.189711 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.189753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.189821 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.190194 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.190197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.190200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:27.190269 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.190271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191599 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.191600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.192647 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.192650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193520 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.193614 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193634 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.193702 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.193763 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260809 00:37:27.194106 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.7.105.65:45057)
0809 00:37:27.190156 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.190161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.190299 (+ 138us) server_negotiation.cc:407] Connection header received
0809 00:37:27.190346 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.190348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.190364 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.190433 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:27.190661 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191339 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193000 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193746 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193820 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193873 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.193972 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.194017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260809 00:37:27.194190 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.7.105.66:44743)
0809 00:37:27.189836 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.189842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.189855 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:37:27.189903 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.189905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.189921 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.190079 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:27.190358 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191494 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.192746 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193418 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193498 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193573 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.194054 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.194102 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
I20260809 00:37:27.194000 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39873 (local address 127.0.0.1:38766)
0809 00:37:27.189932 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.189942 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.189998 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.190242 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.190242 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.190245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:27.190284 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.190285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191569 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.191569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.192904 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.192907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193665 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.193735 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193739 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.193813 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.193859 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":86,"spinlock_wait_cycles":384,"thread_start_us":118,"threads_started":1}
I20260809 00:37:27.194473 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.7.105.67:39873)
0809 00:37:27.189880 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.189884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.190012 (+ 128us) server_negotiation.cc:407] Connection header received
0809 00:37:27.190061 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.190063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.190079 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.190154 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:27.190361 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191461 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193015 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193588 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193662 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193723 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.194317 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.194376 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260809 00:37:27.194240 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.189754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:45057 (local address 127.0.0.1:58986)
0809 00:37:27.190176 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.190186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.190239 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.190519 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.190520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.190523 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:27.190582 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.190583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.191454 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.191455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.192905 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.192907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.193820 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.193821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.193879 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.193884 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.193943 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.194164 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":82,"thread_start_us":65,"threads_started":1}
I20260809 00:37:27.195096 8523 tablet_service.cc:1467] Processing CreateTablet for tablet 17668ad1863d489faa7c63dc6299c539 (DEFAULT_TABLE table=test-table [id=77ca044cebd845049f9c67425073fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:27.195442 8671 tablet_service.cc:1467] Processing CreateTablet for tablet 17668ad1863d489faa7c63dc6299c539 (DEFAULT_TABLE table=test-table [id=77ca044cebd845049f9c67425073fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:27.195591 8523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17668ad1863d489faa7c63dc6299c539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:27.195809 8671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17668ad1863d489faa7c63dc6299c539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:27.196532 8375 tablet_service.cc:1467] Processing CreateTablet for tablet 17668ad1863d489faa7c63dc6299c539 (DEFAULT_TABLE table=test-table [id=77ca044cebd845049f9c67425073fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:27.196888 8375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17668ad1863d489faa7c63dc6299c539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:27.198751 8761 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Bootstrap starting.
I20260809 00:37:27.199378 8762 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: Bootstrap starting.
I20260809 00:37:27.199492 8761 tablet_bootstrap.cc:654] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:27.200232 8762 tablet_bootstrap.cc:654] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:27.200157 8761 log.cc:824] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:27.200840 8762 log.cc:824] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:27.201210 8763 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: Bootstrap starting.
I20260809 00:37:27.201392 8761 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: No bootstrap required, opened a new log
I20260809 00:37:27.201481 8761 ts_tablet_manager.cc:1397] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:37:27.201663 8762 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: No bootstrap required, opened a new log
I20260809 00:37:27.201747 8762 ts_tablet_manager.cc:1397] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:37:27.201917 8763 tablet_bootstrap.cc:654] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:27.202472 8763 log.cc:824] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:27.203301 8763 tablet_bootstrap.cc:492] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: No bootstrap required, opened a new log
I20260809 00:37:27.203375 8763 ts_tablet_manager.cc:1397] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:27.203547 8762 raft_consensus.cc:348] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.203697 8762 raft_consensus.cc:374] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:27.203742 8762 raft_consensus.cc:729] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d26b6aafd64cb397106186cc153d8a, State: Initialized, Role: FOLLOWER
I20260809 00:37:27.203733 8761 raft_consensus.cc:348] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.203919 8761 raft_consensus.cc:374] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:27.203866 8762 consensus_queue.cc:260] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.203994 8761 raft_consensus.cc:729] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cae73f75d5c40549a4fc1070c9bc4f7, State: Initialized, Role: FOLLOWER
I20260809 00:37:27.204141 8761 consensus_queue.cc:260] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.204264 8740 heartbeater.cc:503] Master 127.7.105.126:41993 was elected leader, sending a full tablet report...
I20260809 00:37:27.204270 8762 ts_tablet_manager.cc:1428] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:37:27.204471 8761 ts_tablet_manager.cc:1428] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 00:37:27.204638 8597 heartbeater.cc:503] Master 127.7.105.126:41993 was elected leader, sending a full tablet report...
I20260809 00:37:27.206166 8763 raft_consensus.cc:348] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.206310 8763 raft_consensus.cc:374] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:27.206332 8763 raft_consensus.cc:729] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4b6cf21394fffb740c497088df3fb, State: Initialized, Role: FOLLOWER
I20260809 00:37:27.206463 8763 consensus_queue.cc:260] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.206758 8763 ts_tablet_manager.cc:1428] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:27.206781 8448 heartbeater.cc:503] Master 127.7.105.126:41993 was elected leader, sending a full tablet report...
I20260809 00:37:27.225457 8769 raft_consensus.cc:482] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:27.225592 8769 raft_consensus.cc:504] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.225912 8769 leader_election.cc:288] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d26b6aafd64cb397106186cc153d8a (127.7.105.67:39873), fea4b6cf21394fffb740c497088df3fb (127.7.105.65:45057)
I20260809 00:37:27.229069 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.226116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39873 (local address 127.7.105.66:33771)
0809 00:37:27.226177 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.226190 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.226237 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.226400 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.226402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.226409 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:27.226489 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.226491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.227160 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.227162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.228165 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.228167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.228786 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.228788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.228873 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.228882 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.228949 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.228990 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:37:27.229068 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.226201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:33771 (local address 127.7.105.67:39873)
0809 00:37:27.226262 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.226264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.226274 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:27.226298 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.226298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.226307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.226378 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:27.226592 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.227078 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.228254 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.228704 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.228790 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.228799 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.228956 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.228993 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:37:27.229354 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.226247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:45057 (local address 127.7.105.66:56587)
0809 00:37:27.226454 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:27.226464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:27.226596 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:27.226743 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:27.226744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:27.226745 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:27.226846 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.226848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.227567 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.227568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.228514 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:27.228516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.229086 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.229087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:27.229161 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.229165 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:27.229228 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:37:27.229277 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":126,"threads_started":1}
I20260809 00:37:27.229357 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.226398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:56587 (local address 127.7.105.65:45057)
0809 00:37:27.226454 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.226456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.226570 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:37:27.226657 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.226658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.226668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.226740 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:27.226917 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.227461 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.228604 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.228967 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.229064 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.229072 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.229239 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.229290 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:37:27.229468 8691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17668ad1863d489faa7c63dc6299c539" candidate_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d26b6aafd64cb397106186cc153d8a" is_pre_election: true
I20260809 00:37:27.229665 8691 raft_consensus.cc:2454] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cae73f75d5c40549a4fc1070c9bc4f7 in term 0.
I20260809 00:37:27.229825 8396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17668ad1863d489faa7c63dc6299c539" candidate_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4b6cf21394fffb740c497088df3fb" is_pre_election: true
I20260809 00:37:27.229913 8474 leader_election.cc:302] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cae73f75d5c40549a4fc1070c9bc4f7, f9d26b6aafd64cb397106186cc153d8a; no voters:
I20260809 00:37:27.229988 8396 raft_consensus.cc:2454] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cae73f75d5c40549a4fc1070c9bc4f7 in term 0.
I20260809 00:37:27.230062 8769 raft_consensus.cc:2781] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:27.230122 8769 raft_consensus.cc:482] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:27.230160 8769 raft_consensus.cc:3037] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:27.230860 8769 raft_consensus.cc:504] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.230997 8769 leader_election.cc:288] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [CANDIDATE]: Term 1 election: Requested vote from peers f9d26b6aafd64cb397106186cc153d8a (127.7.105.67:39873), fea4b6cf21394fffb740c497088df3fb (127.7.105.65:45057)
I20260809 00:37:27.231204 8691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17668ad1863d489faa7c63dc6299c539" candidate_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d26b6aafd64cb397106186cc153d8a"
I20260809 00:37:27.231231 8396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17668ad1863d489faa7c63dc6299c539" candidate_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4b6cf21394fffb740c497088df3fb"
I20260809 00:37:27.231287 8691 raft_consensus.cc:3037] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:27.231331 8396 raft_consensus.cc:3037] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:27.232031 8691 raft_consensus.cc:2454] T 17668ad1863d489faa7c63dc6299c539 P f9d26b6aafd64cb397106186cc153d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cae73f75d5c40549a4fc1070c9bc4f7 in term 1.
I20260809 00:37:27.232213 8396 raft_consensus.cc:2454] T 17668ad1863d489faa7c63dc6299c539 P fea4b6cf21394fffb740c497088df3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cae73f75d5c40549a4fc1070c9bc4f7 in term 1.
I20260809 00:37:27.232234 8474 leader_election.cc:302] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cae73f75d5c40549a4fc1070c9bc4f7, f9d26b6aafd64cb397106186cc153d8a; no voters:
I20260809 00:37:27.232398 8769 raft_consensus.cc:2781] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:27.232666 8769 raft_consensus.cc:686] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [term 1 LEADER]: Becoming Leader. State: Replica: 7cae73f75d5c40549a4fc1070c9bc4f7, State: Running, Role: LEADER
I20260809 00:37:27.232795 8769 consensus_queue.cc:237] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } }
I20260809 00:37:27.233693 8238 catalog_manager.cc:5495] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cae73f75d5c40549a4fc1070c9bc4f7 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d26b6aafd64cb397106186cc153d8a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cae73f75d5c40549a4fc1070c9bc4f7" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 44743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fea4b6cf21394fffb740c497088df3fb" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 45057 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
W20260809 00:37:27.240345 8598 tablet.cc:2369] T 17668ad1863d489faa7c63dc6299c539 P 7cae73f75d5c40549a4fc1070c9bc4f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:27.243198 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.235639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.7.105.65:45057)
0809 00:37:27.235689 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.235690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.235795 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:37:27.235850 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.235851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.235860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.235932 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:27.236558 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.237103 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.237769 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.237941 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.237988 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.240306 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:27.240317 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:27.240317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:27.240325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:27.241994 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.242366 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.242370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.242370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.242433 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.242653 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.242654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.242654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.242808 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:27.242973 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.243013 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.243093 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:27.268810 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.261329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.7.105.65:45057)
0809 00:37:27.261417 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.261419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.261528 (+ 109us) server_negotiation.cc:407] Connection header received
0809 00:37:27.261611 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.261612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.261626 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.261698 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:27.262328 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.262816 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.263590 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.263853 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.263930 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.266232 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:27.266233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:27.266234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:27.266241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:27.267620 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.267981 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.267982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.267982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.268115 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.268324 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.268325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.268325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.268427 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:27.268517 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.268679 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.268742 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:27.294123 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:27.287048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.7.105.65:45057)
0809 00:37:27.287141 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:27.287143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:27.287240 (+ 97us) server_negotiation.cc:407] Connection header received
0809 00:37:27.287333 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:27.287334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:27.287351 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:27.287416 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:27.287977 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.288561 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.289336 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:27.289530 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:27.289607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:27.291922 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:27.291924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:27.291924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:27.291931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:27.293183 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.293493 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.293494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.293494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.293545 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:27.293711 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:27.293711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:27.293712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:27.293802 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:27.293874 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:27.293974 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 00:37:27.294031 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 00:37:27.294622 8353 tablet_service.cc:2908] Rejecting scan request for tablet 17668ad1863d489faa7c63dc6299c539: Uninitialized: clean time has not yet been initialized
I20260809 00:37:27.295668 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8306
I20260809 00:37:27.303527 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8454
I20260809 00:37:27.310818 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8602
I20260809 00:37:27.316776 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8203
2026-08-09T00:37:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2129 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8787](info): set up 2 sockets
Aug 09 00:37:27 dist-test-slave-1zqn krb5kdc[8787](info): commencing operation
krb5kdc: starting...
W20260809 00:37:28.415344 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:28 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:28.547295 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36329
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:28.632746 8801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:28.633041 8801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:28.633086 8801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:28.634275 8801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:28.634353 8801 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:37:28.634378 8801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:28.634405 8801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:28.634441 8801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:28.634469 8801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:28.636037 8801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36329
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:28.636356 8801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:28.636615 8801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:28.642089 8810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:28.642084 8807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:28.642129 8801 server_base.cc:1034] running on GCE node
W20260809 00:37:28.642143 8808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:28.642624 8801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:28.642951 8801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:28.644129 8801 hybrid_clock.cc:648] HybridClock initialized: now 1786235848644106 us; error 40 us; skew 500 ppm
Aug 09 00:37:28 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:28.651602 8801 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:28.652240 8801 webserver.cc:460] Webserver started at http://127.7.105.126:36687/ using document root <none> and password file <none>
I20260809 00:37:28.652549 8801 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:28.652652 8801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:28.652804 8801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:28.654081 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "da54b8f5d1684afc865136f34f9ac76c"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "158bbfa14a82257fec638ffd6e48f1dd"
server_key_iv: "991b8a0919bd626c98c15f750a5a8cf3"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.654441 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "da54b8f5d1684afc865136f34f9ac76c"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "158bbfa14a82257fec638ffd6e48f1dd"
server_key_iv: "991b8a0919bd626c98c15f750a5a8cf3"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.656111 8801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:28.656970 8817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:28.657183 8801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:28.657321 8801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "da54b8f5d1684afc865136f34f9ac76c"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "158bbfa14a82257fec638ffd6e48f1dd"
server_key_iv: "991b8a0919bd626c98c15f750a5a8cf3"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.657416 8801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:28.688225 8801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:28.690686 8801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:28.690917 8801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:28.697046 8801 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:36329
I20260809 00:37:28.697105 8883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:36329 every 8 connection(s)
I20260809 00:37:28.697497 8801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:28.698189 8884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:28.700690 8884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Bootstrap starting.
I20260809 00:37:28.701316 8884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:28.701635 8884 log.cc:824] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:28.702343 8884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: No bootstrap required, opened a new log
I20260809 00:37:28.703498 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8801
I20260809 00:37:28.703598 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:28.703718 7589 external_mini_cluster.cc:1428] Setting key 3fa1958b60a80f55c649a5d74462dbf7
I20260809 00:37:28.705122 8884 raft_consensus.cc:348] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } }
I20260809 00:37:28.705303 8884 raft_consensus.cc:374] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:28.705344 8884 raft_consensus.cc:729] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da54b8f5d1684afc865136f34f9ac76c, State: Initialized, Role: FOLLOWER
I20260809 00:37:28.705529 8884 consensus_queue.cc:260] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } }
I20260809 00:37:28.705626 8884 raft_consensus.cc:388] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:28.705650 8884 raft_consensus.cc:482] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:28.705699 8884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:28.706393 8884 raft_consensus.cc:504] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } }
I20260809 00:37:28.706511 8884 leader_election.cc:302] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da54b8f5d1684afc865136f34f9ac76c; no voters:
I20260809 00:37:28.706679 8884 leader_election.cc:288] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:28.706751 8890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:28.706919 8884 sys_catalog.cc:564] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:28.706928 8890 raft_consensus.cc:686] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [term 1 LEADER]: Becoming Leader. State: Replica: da54b8f5d1684afc865136f34f9ac76c, State: Running, Role: LEADER
I20260809 00:37:28.707131 8890 consensus_queue.cc:237] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } }
I20260809 00:37:28.707517 8892 sys_catalog.cc:455] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da54b8f5d1684afc865136f34f9ac76c. Latest consensus state: current_term: 1 leader_uuid: "da54b8f5d1684afc865136f34f9ac76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } } }
I20260809 00:37:28.707631 8892 sys_catalog.cc:458] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:28.707517 8891 sys_catalog.cc:455] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da54b8f5d1684afc865136f34f9ac76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da54b8f5d1684afc865136f34f9ac76c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36329 } } }
I20260809 00:37:28.707698 8891 sys_catalog.cc:458] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:28.707942 8899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:28.708513 8899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:28.710460 8899 catalog_manager.cc:1347] Generated new cluster ID: 3bf753dcde0044158beaaa8e8bbeb8c9
I20260809 00:37:28.710516 8899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:28 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:28.716133 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:28.704314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.7.105.126:36329)
0809 00:37:28.704551 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:28.704564 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:28.704606 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:37:28.705083 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:28.705093 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:28.705159 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:28.705317 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:28.706032 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.707572 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.708769 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.709365 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.709488 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:28.712490 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:28.712494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:28.712494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:28.712506 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:28.714452 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.714875 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.714878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.714878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.714999 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.715235 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.715235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.715236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.715519 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:28.715672 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:28.715780 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:37:28.715909 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
I20260809 00:37:28.719115 8899 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:28.719636 8899 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:28.724328 8899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Generated new TSK 0
I20260809 00:37:28.724512 8899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:28.759357 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36329
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:28.845180 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:28.845389 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:28.845417 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:28.846554 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:28.846590 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:28.846611 8912 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:28.846627 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:28.848032 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:28.848307 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:28.848510 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:28.853750 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:28.853768 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:28.853791 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:28.854104 8912 server_base.cc:1034] running on GCE node
I20260809 00:37:28.854308 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:28.854558 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:28.855726 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786235848855737 us; error 59 us; skew 500 ppm
Aug 09 00:37:28 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:28.858614 8912 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:28.859122 8912 webserver.cc:460] Webserver started at http://127.7.105.65:46299/ using document root <none> and password file <none>
I20260809 00:37:28.859367 8912 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:28.859413 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:28.859579 8912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:28.860831 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "f95c83cb078a4e5889ed5a9552238713"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "92674289ba89e6d6ce6c537895536e23"
server_key_iv: "49193e96050b27e32201c2f123cfe710"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.861198 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "f95c83cb078a4e5889ed5a9552238713"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "92674289ba89e6d6ce6c537895536e23"
server_key_iv: "49193e96050b27e32201c2f123cfe710"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.862680 8912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:28.863444 8929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:28.863673 8912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:28.863770 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "f95c83cb078a4e5889ed5a9552238713"
format_stamp: "Formatted at 2026-08-09 00:37:28 on dist-test-slave-1zqn"
server_key: "92674289ba89e6d6ce6c537895536e23"
server_key_iv: "49193e96050b27e32201c2f123cfe710"
server_key_version: "encryptionkey@0"
I20260809 00:37:28.863842 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:28.881666 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:28.883991 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:28.884253 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:28.884833 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:28.884868 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:28.884892 8912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:28.884940 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:28.892416 8912 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:43399
I20260809 00:37:28.892815 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:28 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:28.893122 9050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:43399 every 8 connection(s)
I20260809 00:37:28.895493 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 8912
I20260809 00:37:28.895604 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:28.895817 7589 external_mini_cluster.cc:1428] Setting key b84d68a390a3ccfce4467952bf794409
I20260809 00:37:28.896422 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:28.885612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:45259 (local address 127.7.105.126:36329)
0809 00:37:28.885688 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:28.885690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:28.885918 (+ 228us) server_negotiation.cc:407] Connection header received
0809 00:37:28.886674 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:28.886676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:28.886695 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:28.886781 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:28.887754 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.888635 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.890378 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.890593 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.890654 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:28.893534 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:28.893536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:28.893536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:28.893548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:28.894829 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.895225 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.895226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.895226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.895278 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.895604 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.895605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.895605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.895747 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:28.895801 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:28.896235 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 00:37:28.896325 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:37:28.896596 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:28.885587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.65:45259)
0809 00:37:28.885789 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:28.885826 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:28.886563 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:28.886899 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:28.886905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:28.887340 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:28.887657 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:28.887666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.888760 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.888761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:28.890272 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:28.890276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.890691 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.890692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:28.890851 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:28.891543 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:28.891568 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:28.893446 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:28.894906 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:28.894913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:28.894915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:28.895158 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:28.895364 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:28.895365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:28.895366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:28.895514 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:28.895816 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:28.895824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:28.895995 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 00:37:28.896190 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 00:37:28.896335 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260809 00:37:28.897104 9051 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36329
I20260809 00:37:28.897207 9051 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:28.897418 9051 heartbeater.cc:511] Master 127.7.105.126:36329 requested a full tablet report, sending...
I20260809 00:37:28.897926 8834 ts_manager.cc:194] Registered new tserver with Master: f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399)
I20260809 00:37:28.898803 8834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:45259
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:37:28.905586 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:28.899686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43175 (local address 127.7.105.126:36329)
0809 00:37:28.899728 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:28.899729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:28.899914 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:37:28.899975 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:28.899976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:28.899987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:28.900036 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:28.900360 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.900833 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.901513 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.901719 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.901796 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:28.902725 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:28.902727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:28.902727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:28.902734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:28.904309 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.904751 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.904752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.904753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.904813 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:28.905003 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:28.905004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:28.905005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:28.905119 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:28.905191 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:28.905370 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:37:28.905472 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:28.905635 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:28.899686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.65:43175)
0809 00:37:28.899833 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:28.899849 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:28.899925 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:28.900044 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:28.900045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:28.900225 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:28.900282 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:28.900283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.900929 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.900930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:28.901429 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:28.901430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:28.901794 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:28.901795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:28.901869 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:28.902318 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:28.902322 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:28.902621 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:28.904463 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:28.904464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:28.904464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:28.904673 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:28.904884 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:28.904884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:28.904885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:28.904948 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:28.905196 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:28.905197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:28.905257 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:37:28.905332 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:37:28.905500 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:28.923871 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36329
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:29.009526 9062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:29.009747 9062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:29.009778 9062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:29.010982 9062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:29.011029 9062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:29.011050 9062 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:29.011070 9062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:29.012611 9062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:29.012884 9062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:29.013149 9062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:29.018463 9071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:29.018530 9069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:29.018826 9062 server_base.cc:1034] running on GCE node
W20260809 00:37:29.018442 9068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:29.019124 9062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:29.019381 9062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:29.020519 9062 hybrid_clock.cc:648] HybridClock initialized: now 1786235849020496 us; error 41 us; skew 500 ppm
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:29.023113 9062 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:29.023594 9062 webserver.cc:460] Webserver started at http://127.7.105.66:37601/ using document root <none> and password file <none>
I20260809 00:37:29.023839 9062 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:29.023911 9062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:29.024111 9062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:29.025215 9062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "1e05fcc69ce5458f8eb45f2c703922a5"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "45ac19a7d6f38eac82c6d5700cfcc912"
server_key_iv: "1db6a98307ee707ec0d6850c0ff21f38"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.025586 9062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "1e05fcc69ce5458f8eb45f2c703922a5"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "45ac19a7d6f38eac82c6d5700cfcc912"
server_key_iv: "1db6a98307ee707ec0d6850c0ff21f38"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.026958 9062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:29.027727 9079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.027905 9062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:29.028010 9062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "1e05fcc69ce5458f8eb45f2c703922a5"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "45ac19a7d6f38eac82c6d5700cfcc912"
server_key_iv: "1db6a98307ee707ec0d6850c0ff21f38"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.028131 9062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:29.041783 9062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:29.044042 9062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:29.044299 9062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:29.044874 9062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:29.044934 9062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.044999 9062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:29.045030 9062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:29.052484 9062 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:34317
I20260809 00:37:29.052845 9062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:29.052839 9203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:34317 every 8 connection(s)
I20260809 00:37:29.055310 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.045306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:59179 (local address 127.7.105.126:36329)
0809 00:37:29.045376 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.045379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.045638 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:37:29.046119 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.046120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.046133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.046216 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.046897 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.047748 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.048856 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.049198 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.049281 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.052283 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.052286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.052286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.052294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.053892 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.054299 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.054300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.054300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.054358 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.054643 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.054645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.054645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.054771 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.054840 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.055124 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.055215 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:37:29.055485 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.045310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.66:59179)
0809 00:37:29.045481 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.045523 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.046013 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.046287 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.046291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.046548 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:29.046796 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.046802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.047851 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.047852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.048762 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.048765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.049292 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.049293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.049471 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.050081 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:29.050091 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:29.052187 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:29.053988 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.053996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.053998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.054221 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.054436 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.054436 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.054437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.054566 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.054843 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:29.054851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:29.054988 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.055099 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.055257 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260809 00:37:29.056017 9205 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36329
I20260809 00:37:29.056146 9205 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:29.056334 9205 heartbeater.cc:511] Master 127.7.105.126:36329 requested a full tablet report, sending...
I20260809 00:37:29.056726 8835 ts_manager.cc:194] Registered new tserver with Master: 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317)
I20260809 00:37:29.057072 8835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:59179
I20260809 00:37:29.059620 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9062
I20260809 00:37:29.059721 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:29.059911 7589 external_mini_cluster.cc:1428] Setting key 6f86338dfcd9a486a8ecff5a26d6e338
I20260809 00:37:29.062974 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.057186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:60939 (local address 127.7.105.126:36329)
0809 00:37:29.057223 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.057224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.057380 (+ 156us) server_negotiation.cc:407] Connection header received
0809 00:37:29.057406 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.057407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.057418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.057478 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.057805 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.058368 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.059074 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.059250 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.059301 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.060244 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.060246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.060246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.060253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.061811 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.062150 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.062151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.062152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.062207 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.062421 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.062422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.062422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.062574 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.062653 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.062815 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.062884 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:29.062989 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.057175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.66:60939)
0809 00:37:29.057307 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.057317 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.057370 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.057476 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.057477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.057676 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:29.057733 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.057735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.058465 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.058466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.058992 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.058994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.059316 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.059317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.059373 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.059788 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:29.059793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:29.060160 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:29.061899 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.061900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.061901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.062076 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.062282 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.062283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.062283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.062342 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.062645 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:29.062646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:29.062693 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.062777 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.062882 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:29.089604 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36329
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:29.176290 9215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:29.176512 9215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:29.176539 9215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:29.177687 9215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:29.177729 9215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:29.177752 9215 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:29.177769 9215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:29.179226 9215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:29.179438 9215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:29.179622 9215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:29.184957 9221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:29.184954 9220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:29.185050 9215 server_base.cc:1034] running on GCE node
W20260809 00:37:29.184954 9223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:29.185556 9215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:29.185770 9215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:29.186940 9215 hybrid_clock.cc:648] HybridClock initialized: now 1786235849186921 us; error 33 us; skew 500 ppm
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:29.189714 9215 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:29.190272 9215 webserver.cc:460] Webserver started at http://127.7.105.67:43385/ using document root <none> and password file <none>
I20260809 00:37:29.190568 9215 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:29.190655 9215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:29.190837 9215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:29.192180 9215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "dc85883e044746a8a14cbf0aaf889c89"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "4d8d354d8e6701261521b8b0341fdafb"
server_key_iv: "078907786d23249f5a0a0defef9eb4c2"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.192631 9215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "dc85883e044746a8a14cbf0aaf889c89"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "4d8d354d8e6701261521b8b0341fdafb"
server_key_iv: "078907786d23249f5a0a0defef9eb4c2"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.194236 9215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:37:29.195125 9231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.195334 9215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:29.195469 9215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "dc85883e044746a8a14cbf0aaf889c89"
format_stamp: "Formatted at 2026-08-09 00:37:29 on dist-test-slave-1zqn"
server_key: "4d8d354d8e6701261521b8b0341fdafb"
server_key_iv: "078907786d23249f5a0a0defef9eb4c2"
server_key_version: "encryptionkey@0"
I20260809 00:37:29.195559 9215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:29.247233 9215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:29.250393 9215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:29.250587 9215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:29.251240 9215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:29.251312 9215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.251380 9215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:29.251441 9215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:29.260915 9215 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:43391
I20260809 00:37:29.261025 9361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:43391 every 8 connection(s)
I20260809 00:37:29.261428 9215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:29.263109 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.251922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:39967 (local address 127.7.105.126:36329)
0809 00:37:29.252013 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.252016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.252217 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:37:29.252734 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.252736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.252753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.252854 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.253530 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.254490 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.256157 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.256436 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.256528 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.259840 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.259842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.259842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.259851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.261696 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.262191 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.262192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.262193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.262279 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.262549 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.262550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.262550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.262674 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.262740 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.262962 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.263037 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:37:29.263245 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.251842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.67:39967)
0809 00:37:29.252045 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.252116 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.252624 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.252961 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.252965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.253227 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:29.253437 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.253442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.254618 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.254619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.256025 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.256029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.256540 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.256541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.256690 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.257254 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:29.257265 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:29.259732 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:29.261808 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.261816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.261818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.262088 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.262362 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.262363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.262363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.262493 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.262737 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:29.262742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:29.262857 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.262935 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.263058 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260809 00:37:29.263751 9362 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36329
I20260809 00:37:29.263898 9362 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:29.264194 9362 heartbeater.cc:511] Master 127.7.105.126:36329 requested a full tablet report, sending...
I20260809 00:37:29.264663 8835 ts_manager.cc:194] Registered new tserver with Master: dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67:43391)
I20260809 00:37:29.265038 8835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:39967
I20260809 00:37:29.266104 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9215
I20260809 00:37:29.266208 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:29.266369 7589 external_mini_cluster.cc:1428] Setting key 67a71f67a44d2b0c3f0b929a1e35f0d1
I20260809 00:37:29.267321 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:29.271795 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.265110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:55563 (local address 127.7.105.126:36329)
0809 00:37:29.265160 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.265163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.265257 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:37:29.265326 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.265327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.265338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.265387 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.265833 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.266514 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.267608 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.267943 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.268037 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.269003 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.269005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.269005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.269014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.270628 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.271033 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.271034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.271034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.271094 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.271307 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.271307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.271308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.271443 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.271505 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.271663 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.271728 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:37:29.271912 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.264945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.7.105.67:55563)
0809 00:37:29.265186 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.265198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.265271 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.265445 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.265446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.265677 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:29.265747 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.265749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.266654 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.266655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.267493 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.267496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.268032 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.268032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.268161 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.268637 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:29.268641 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:29.268907 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:29.270717 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.270718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.270718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.270953 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.271167 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.271168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.271168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.271238 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.271513 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:29.271514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:29.271573 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.271646 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.271762 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:29.293154 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.285991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.7.105.126:36329)
0809 00:37:29.286075 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.286077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.286088 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:29.286134 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.286135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.286146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.286218 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.286824 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.287358 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.288141 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.288341 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.288401 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.290685 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.290686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.290686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.290693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.292164 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.292462 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.292463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.292463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.292512 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.292678 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.292678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.292679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.292775 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.292850 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.292934 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.292992 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:37:29.294726 8835 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:29.296356 8835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:29.304471 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.299166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43399 (local address 127.0.0.1:46154)
0809 00:37:29.299397 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.299408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.299460 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.299893 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.299894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.299896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.299935 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.299937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.302264 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.302265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.303120 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.303122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.304149 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.304150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.304238 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.304253 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.304333 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.304388 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":75,"threads_started":1}
I20260809 00:37:29.304559 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.299388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.7.105.65:43399)
0809 00:37:29.299725 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.299730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.299740 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:29.299782 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.299784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.299801 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.299881 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.301309 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.302177 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.303234 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.303958 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.304029 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.304109 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.304366 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.304428 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":112,"threads_started":1}
I20260809 00:37:29.305126 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.298906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:34317 (local address 127.0.0.1:34224)
0809 00:37:29.299096 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.299140 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.299210 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.299818 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.299819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.299821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.299901 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.299903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.301295 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.301296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.302742 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.302744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.304846 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.304847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.304925 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.304930 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.304993 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.305049 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":120,"thread_start_us":74,"threads_started":1}
I20260809 00:37:29.305204 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.299052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.7.105.66:34317)
0809 00:37:29.299478 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.299484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.299495 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:29.299542 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.299545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.299562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.299647 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.299991 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.301181 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.303951 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.304773 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.304846 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.304899 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.305014 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.305084 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":109,"threads_started":1}
I20260809 00:37:29.305922 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.298911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:43391 (local address 127.0.0.1:42832)
0809 00:37:29.299389 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.299401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.299458 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.299773 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.299776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.299779 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.299938 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.299940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.301292 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.301293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.302741 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.302744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.305694 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.305695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.305765 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.305768 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.305815 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.305852 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":95,"thread_start_us":88,"threads_started":1}
I20260809 00:37:29.306202 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.299107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.7.105.67:43391)
0809 00:37:29.299338 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.299344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.299527 (+ 183us) server_negotiation.cc:407] Connection header received
0809 00:37:29.299584 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.299588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.299603 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.299670 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.300006 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.301210 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.302835 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.303736 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.303839 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.303909 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.306038 (+ 2129us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.306098 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
I20260809 00:37:29.306314 9132 tablet_service.cc:1467] Processing CreateTablet for tablet 50071c9651904812bf401f72d0673298 (DEFAULT_TABLE table=test-table [id=96d50e6924a94a28b2a318a14a346f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.306718 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50071c9651904812bf401f72d0673298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.308461 9286 tablet_service.cc:1467] Processing CreateTablet for tablet 50071c9651904812bf401f72d0673298 (DEFAULT_TABLE table=test-table [id=96d50e6924a94a28b2a318a14a346f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.308589 8984 tablet_service.cc:1467] Processing CreateTablet for tablet 50071c9651904812bf401f72d0673298 (DEFAULT_TABLE table=test-table [id=96d50e6924a94a28b2a318a14a346f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.308804 9286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50071c9651904812bf401f72d0673298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.308959 8984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50071c9651904812bf401f72d0673298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.310737 9386 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Bootstrap starting.
I20260809 00:37:29.311399 9386 tablet_bootstrap.cc:654] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.311403 9387 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Bootstrap starting.
I20260809 00:37:29.311728 9386 log.cc:824] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:29.312394 9387 tablet_bootstrap.cc:654] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.312640 9386 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: No bootstrap required, opened a new log
I20260809 00:37:29.312709 9386 ts_tablet_manager.cc:1397] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:29.312747 9387 log.cc:824] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:29.312958 9389 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Bootstrap starting.
I20260809 00:37:29.313551 9387 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: No bootstrap required, opened a new log
I20260809 00:37:29.313622 9389 tablet_bootstrap.cc:654] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.313637 9387 ts_tablet_manager.cc:1397] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:29.313939 9389 log.cc:824] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:29.314592 9386 raft_consensus.cc:348] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.314745 9389 tablet_bootstrap.cc:492] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: No bootstrap required, opened a new log
I20260809 00:37:29.314764 9386 raft_consensus.cc:374] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.314811 9386 raft_consensus.cc:729] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e05fcc69ce5458f8eb45f2c703922a5, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.314807 9389 ts_tablet_manager.cc:1397] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:29.314963 9386 consensus_queue.cc:260] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.315271 9386 ts_tablet_manager.cc:1428] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:29.315415 9205 heartbeater.cc:503] Master 127.7.105.126:36329 was elected leader, sending a full tablet report...
I20260809 00:37:29.315886 9387 raft_consensus.cc:348] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.316043 9387 raft_consensus.cc:374] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.316139 9387 raft_consensus.cc:729] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc85883e044746a8a14cbf0aaf889c89, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.316262 9387 consensus_queue.cc:260] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.316658 9362 heartbeater.cc:503] Master 127.7.105.126:36329 was elected leader, sending a full tablet report...
I20260809 00:37:29.316913 9387 ts_tablet_manager.cc:1428] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 00:37:29.316912 9389 raft_consensus.cc:348] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.317052 9389 raft_consensus.cc:374] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.317099 9389 raft_consensus.cc:729] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95c83cb078a4e5889ed5a9552238713, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.317214 9389 consensus_queue.cc:260] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.317507 9389 ts_tablet_manager.cc:1428] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:29.317593 9051 heartbeater.cc:503] Master 127.7.105.126:36329 was elected leader, sending a full tablet report...
W20260809 00:37:29.393600 9052 tablet.cc:2369] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:29.484447 9394 raft_consensus.cc:482] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:29.484637 9394 raft_consensus.cc:504] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.485152 9394 leader_election.cc:288] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317), f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399)
I20260809 00:37:29.488389 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.485633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:43101 (local address 127.7.105.65:43399)
0809 00:37:29.485680 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.485682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.485843 (+ 161us) server_negotiation.cc:407] Connection header received
0809 00:37:29.485899 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.485900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.485910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.485965 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.486152 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.486651 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.487638 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.487979 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.488059 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.488094 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.488263 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.488313 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:29.488387 9396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.485519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43399 (local address 127.7.105.67:43101)
0809 00:37:29.485753 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.485764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.485841 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.486014 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.486015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.486016 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.486084 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.486086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.486743 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.486744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.487483 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.487484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.488098 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.488099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.488193 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.488196 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.488239 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.488290 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":150,"threads_started":1}
I20260809 00:37:29.488442 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.485480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:50399 (local address 127.7.105.66:34317)
0809 00:37:29.485529 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.485532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.485544 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:37:29.485578 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.485580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.485590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.485713 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.486024 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.486567 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.487553 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.487977 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.488066 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.488097 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.488317 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.488364 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:29.488397 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.485390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:34317 (local address 127.7.105.67:50399)
0809 00:37:29.485452 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.485465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.485513 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.485737 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.485738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.485740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.485881 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.485883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.486649 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.486650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.487449 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.487452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.488109 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.488110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.488213 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.488215 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.488292 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.488329 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:37:29.489025 9149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50071c9651904812bf401f72d0673298" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" is_pre_election: true
I20260809 00:37:29.489022 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50071c9651904812bf401f72d0673298" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713" is_pre_election: true
I20260809 00:37:29.489187 9149 raft_consensus.cc:2454] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 0.
I20260809 00:37:29.489187 9004 raft_consensus.cc:2454] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 0.
I20260809 00:37:29.489415 9233 leader_election.cc:302] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e05fcc69ce5458f8eb45f2c703922a5, dc85883e044746a8a14cbf0aaf889c89; no voters:
I20260809 00:37:29.489547 9394 raft_consensus.cc:2781] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:29.489634 9394 raft_consensus.cc:482] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:29.489671 9394 raft_consensus.cc:3037] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.490402 9394 raft_consensus.cc:504] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.490549 9394 leader_election.cc:288] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 election: Requested vote from peers 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317), f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399)
I20260809 00:37:29.490741 9149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50071c9651904812bf401f72d0673298" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e05fcc69ce5458f8eb45f2c703922a5"
I20260809 00:37:29.490774 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50071c9651904812bf401f72d0673298" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713"
I20260809 00:37:29.490826 9149 raft_consensus.cc:3037] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.490866 9004 raft_consensus.cc:3037] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.491570 9149 raft_consensus.cc:2454] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 1.
I20260809 00:37:29.491585 9004 raft_consensus.cc:2454] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 1.
I20260809 00:37:29.491739 9233 leader_election.cc:302] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e05fcc69ce5458f8eb45f2c703922a5, dc85883e044746a8a14cbf0aaf889c89; no voters:
I20260809 00:37:29.491847 9394 raft_consensus.cc:2781] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:29.492004 9394 raft_consensus.cc:686] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 1 LEADER]: Becoming Leader. State: Replica: dc85883e044746a8a14cbf0aaf889c89, State: Running, Role: LEADER
I20260809 00:37:29.492288 9394 consensus_queue.cc:237] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } }
I20260809 00:37:29.493191 8835 catalog_manager.cc:5495] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } health_report { overall_health: UNKNOWN } } }
W20260809 00:37:29.512384 9363 tablet.cc:2369] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:29.529922 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.527511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.7.105.67:43391)
0809 00:37:29.527581 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.527584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.527594 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:29.527691 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.527692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.527709 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.527770 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:29.527935 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.528411 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.529171 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.529346 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.529395 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.529583 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.529671 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.529727 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.529773 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.529791 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:37:29.531952 9149 raft_consensus.cc:1261] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Refusing update from remote peer dc85883e044746a8a14cbf0aaf889c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.531952 9004 raft_consensus.cc:1261] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Refusing update from remote peer dc85883e044746a8a14cbf0aaf889c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.532348 9394 consensus_queue.cc:1035] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.532568 9397 consensus_queue.cc:1035] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.535533 9403 mvcc.cc:205] Tried to move back new op lower bound from 7316422039679549440 to 7316422039521153024. Current Snapshot: MvccSnapshot[applied={T|T < 7316422039679549440}]
I20260809 00:37:29.536339 9402 mvcc.cc:205] Tried to move back new op lower bound from 7316422039679549440 to 7316422039521153024. Current Snapshot: MvccSnapshot[applied={T|T < 7316422039679549440}]
I20260809 00:37:29.539383 8834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:29.539490 8834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:29.540599 8834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Sending DeleteTablet for 3 replicas of tablet 50071c9651904812bf401f72d0673298
I20260809 00:37:29.540999 9132 tablet_service.cc:1514] Processing DeleteTablet for tablet 50071c9651904812bf401f72d0673298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:34224
I20260809 00:37:29.541007 8984 tablet_service.cc:1514] Processing DeleteTablet for tablet 50071c9651904812bf401f72d0673298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46154
I20260809 00:37:29.541070 9286 tablet_service.cc:1514] Processing DeleteTablet for tablet 50071c9651904812bf401f72d0673298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:42832
I20260809 00:37:29.541292 9411 tablet_replica.cc:330] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: stopping tablet replica
I20260809 00:37:29.541414 9411 raft_consensus.cc:2229] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:29.541522 9411 raft_consensus.cc:2258] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.541531 9412 tablet_replica.cc:330] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: stopping tablet replica
I20260809 00:37:29.541503 8834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:37:29.541625 9412 raft_consensus.cc:2229] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 00:37:29.541656 8834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:29.541688 9413 tablet_replica.cc:330] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: stopping tablet replica
I20260809 00:37:29.541714 9412 raft_consensus.cc:2258] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.541769 9413 raft_consensus.cc:2229] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:29.541857 9413 raft_consensus.cc:2258] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.542333 9413 ts_tablet_manager.cc:1905] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.542958 8984 tablet_service.cc:1467] Processing CreateTablet for tablet ab8a07cabd8a462082e57d51e206297c (DEFAULT_TABLE table=test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.543095 8984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab8a07cabd8a462082e57d51e206297c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.543100 9132 tablet_service.cc:1467] Processing CreateTablet for tablet ab8a07cabd8a462082e57d51e206297c (DEFAULT_TABLE table=test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.543198 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab8a07cabd8a462082e57d51e206297c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.544034 9413 ts_tablet_manager.cc:1918] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:29.544132 9413 log.cc:1196] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/50071c9651904812bf401f72d0673298
I20260809 00:37:29.544391 9413 ts_tablet_manager.cc:1939] T 50071c9651904812bf401f72d0673298 P dc85883e044746a8a14cbf0aaf889c89: Deleting consensus metadata
I20260809 00:37:29.544557 9389 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Bootstrap starting.
I20260809 00:37:29.544683 9286 tablet_service.cc:1467] Processing CreateTablet for tablet ab8a07cabd8a462082e57d51e206297c (DEFAULT_TABLE table=test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:29.544791 9286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab8a07cabd8a462082e57d51e206297c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.544963 9389 tablet_bootstrap.cc:654] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.545671 9389 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: No bootstrap required, opened a new log
I20260809 00:37:29.545727 9389 ts_tablet_manager.cc:1397] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:29.545892 9389 raft_consensus.cc:348] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.545966 9389 raft_consensus.cc:374] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.545956 9387 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Bootstrap starting.
I20260809 00:37:29.545989 9389 raft_consensus.cc:729] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95c83cb078a4e5889ed5a9552238713, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.545997 9386 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Bootstrap starting.
I20260809 00:37:29.546041 9389 consensus_queue.cc:260] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.546154 9389 ts_tablet_manager.cc:1428] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:29.546432 9387 tablet_bootstrap.cc:654] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.546639 9411 ts_tablet_manager.cc:1905] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.546733 9386 tablet_bootstrap.cc:654] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.547362 9386 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: No bootstrap required, opened a new log
I20260809 00:37:29.547410 9386 ts_tablet_manager.cc:1397] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:29.547583 9386 raft_consensus.cc:348] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.547672 9386 raft_consensus.cc:374] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.547700 9386 raft_consensus.cc:729] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e05fcc69ce5458f8eb45f2c703922a5, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.547742 9386 consensus_queue.cc:260] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.547835 9386 ts_tablet_manager.cc:1428] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.548388 9411 ts_tablet_manager.cc:1918] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:29.548408 8821 catalog_manager.cc:4841] TS dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67:43391): tablet 50071c9651904812bf401f72d0673298 (table test-table [id=96d50e6924a94a28b2a318a14a346f57]) successfully deleted
I20260809 00:37:29.548465 9411 log.cc:1196] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/50071c9651904812bf401f72d0673298
I20260809 00:37:29.548595 9412 ts_tablet_manager.cc:1905] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.548722 9411 ts_tablet_manager.cc:1939] T 50071c9651904812bf401f72d0673298 P f95c83cb078a4e5889ed5a9552238713: Deleting consensus metadata
I20260809 00:37:29.549161 8819 catalog_manager.cc:4841] TS f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399): tablet 50071c9651904812bf401f72d0673298 (table test-table [id=96d50e6924a94a28b2a318a14a346f57]) successfully deleted
I20260809 00:37:29.550256 9412 ts_tablet_manager.cc:1918] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:29.550335 9412 log.cc:1196] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/50071c9651904812bf401f72d0673298
I20260809 00:37:29.550362 9387 tablet_bootstrap.cc:492] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: No bootstrap required, opened a new log
I20260809 00:37:29.550413 9387 ts_tablet_manager.cc:1397] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260809 00:37:29.550623 9412 ts_tablet_manager.cc:1939] T 50071c9651904812bf401f72d0673298 P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting consensus metadata
I20260809 00:37:29.550565 9387 raft_consensus.cc:348] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.550662 9387 raft_consensus.cc:374] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.550691 9387 raft_consensus.cc:729] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc85883e044746a8a14cbf0aaf889c89, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.550746 9387 consensus_queue.cc:260] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.550834 9387 ts_tablet_manager.cc:1428] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.551076 8819 catalog_manager.cc:4841] TS 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317): tablet 50071c9651904812bf401f72d0673298 (table test-table [id=96d50e6924a94a28b2a318a14a346f57]) successfully deleted
W20260809 00:37:29.553789 9206 tablet.cc:2369] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:29.649538 9392 raft_consensus.cc:482] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:29.649658 9392 raft_consensus.cc:504] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.650027 9392 leader_election.cc:288] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399), dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67:43391)
I20260809 00:37:29.653908 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.650315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:58063 (local address 127.7.105.65:43399)
0809 00:37:29.650366 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.650368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.650491 (+ 123us) server_negotiation.cc:407] Connection header received
0809 00:37:29.650648 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.650650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.650661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.650730 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.650908 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.651722 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.652992 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.653553 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.653627 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.653636 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.653781 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.653829 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:37:29.653910 9417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.650245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43399 (local address 127.7.105.66:58063)
0809 00:37:29.650421 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.650433 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.650523 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.650761 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.650762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.650765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.650840 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.650843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.652151 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.652152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.652896 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.652898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.653635 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.653636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.653712 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.653717 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.653776 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.653821 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260809 00:37:29.653987 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.650445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:39047 (local address 127.7.105.67:43391)
0809 00:37:29.650513 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.650516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.650775 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:37:29.650802 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.650803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.650813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.650859 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:29.651044 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.651725 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.653120 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.653650 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.653710 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.653717 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.653881 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.653917 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:37:29.654270 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.650245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:43391 (local address 127.7.105.66:39047)
0809 00:37:29.650540 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.650549 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.650601 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.650906 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.650907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.650909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:29.650968 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.650970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.651804 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.651805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.653033 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.653036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.653713 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.653714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.653794 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.653798 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.654097 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.654165 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":19,"thread_start_us":58,"threads_started":1}
I20260809 00:37:29.654421 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab8a07cabd8a462082e57d51e206297c" candidate_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713" is_pre_election: true
I20260809 00:37:29.654533 9004 raft_consensus.cc:2454] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e05fcc69ce5458f8eb45f2c703922a5 in term 0.
I20260809 00:37:29.654507 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab8a07cabd8a462082e57d51e206297c" candidate_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc85883e044746a8a14cbf0aaf889c89" is_pre_election: true
I20260809 00:37:29.654651 9309 raft_consensus.cc:2454] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e05fcc69ce5458f8eb45f2c703922a5 in term 0.
I20260809 00:37:29.654729 9081 leader_election.cc:302] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e05fcc69ce5458f8eb45f2c703922a5, f95c83cb078a4e5889ed5a9552238713; no voters:
I20260809 00:37:29.654871 9392 raft_consensus.cc:2781] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:29.654910 9392 raft_consensus.cc:482] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:29.654942 9392 raft_consensus.cc:3037] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.655648 9392 raft_consensus.cc:504] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.655807 9392 leader_election.cc:288] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [CANDIDATE]: Term 1 election: Requested vote from peers f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399), dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67:43391)
I20260809 00:37:29.655972 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab8a07cabd8a462082e57d51e206297c" candidate_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713"
I20260809 00:37:29.656100 9004 raft_consensus.cc:3037] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.656029 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab8a07cabd8a462082e57d51e206297c" candidate_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc85883e044746a8a14cbf0aaf889c89"
I20260809 00:37:29.656143 9309 raft_consensus.cc:3037] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.656692 9004 raft_consensus.cc:2454] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e05fcc69ce5458f8eb45f2c703922a5 in term 1.
I20260809 00:37:29.656867 9081 leader_election.cc:302] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e05fcc69ce5458f8eb45f2c703922a5, f95c83cb078a4e5889ed5a9552238713; no voters:
I20260809 00:37:29.656957 9392 raft_consensus.cc:2781] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:29.656949 9309 raft_consensus.cc:2454] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e05fcc69ce5458f8eb45f2c703922a5 in term 1.
I20260809 00:37:29.657070 9392 raft_consensus.cc:686] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 LEADER]: Becoming Leader. State: Replica: 1e05fcc69ce5458f8eb45f2c703922a5, State: Running, Role: LEADER
I20260809 00:37:29.657145 9392 consensus_queue.cc:237] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.657948 8834 catalog_manager.cc:5495] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235848, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:29.687837 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.681014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.7.105.126:36329)
0809 00:37:29.681116 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.681120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.681242 (+ 122us) server_negotiation.cc:407] Connection header received
0809 00:37:29.681287 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.681288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.681298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.681351 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.681870 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.682369 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.683041 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.683233 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.683285 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.685576 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.685578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.685578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.685584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.686872 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.687225 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.687226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.687226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.687275 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.687441 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.687442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.687442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.687541 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.687590 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.687704 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.687770 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:37:29.687996 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.680960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36329 (local address 127.0.0.1:58672)
0809 00:37:29.681164 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.681174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.681240 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.681366 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.681366 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.681725 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:29.681800 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.681802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.682438 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.682439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.682969 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.682970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.683302 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.683303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.683351 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.683931 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:29.683936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:29.685487 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:29.686954 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.686960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.686963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.687160 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.687333 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:29.687334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:29.687334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:29.687391 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:29.687594 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:29.687600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:29.687639 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.687690 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.687842 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":59,"thread_start_us":131,"threads_started":1}
I20260809 00:37:29.690132 8834 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:58672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:37:29.690464 8834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:29.692003 9132 tablet_service.cc:1467] Processing CreateTablet for tablet adfbfe33344f4f48b9387ba16ebe7a30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0cf2a56edd04ed38b33afdb5920fa8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:29.691967 9286 tablet_service.cc:1467] Processing CreateTablet for tablet adfbfe33344f4f48b9387ba16ebe7a30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0cf2a56edd04ed38b33afdb5920fa8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:29.692222 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfbfe33344f4f48b9387ba16ebe7a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.692224 9286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfbfe33344f4f48b9387ba16ebe7a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.692247 8984 tablet_service.cc:1467] Processing CreateTablet for tablet adfbfe33344f4f48b9387ba16ebe7a30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0cf2a56edd04ed38b33afdb5920fa8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:29.692387 8984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfbfe33344f4f48b9387ba16ebe7a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:29.693639 9387 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: Bootstrap starting.
I20260809 00:37:29.693851 9389 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: Bootstrap starting.
I20260809 00:37:29.694000 9387 tablet_bootstrap.cc:654] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.694278 9389 tablet_bootstrap.cc:654] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.694631 9386 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: Bootstrap starting.
I20260809 00:37:29.694661 9387 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: No bootstrap required, opened a new log
I20260809 00:37:29.694700 9387 ts_tablet_manager.cc:1397] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:29.694813 9387 raft_consensus.cc:348] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.694888 9387 raft_consensus.cc:374] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.694892 9389 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: No bootstrap required, opened a new log
I20260809 00:37:29.694911 9387 raft_consensus.cc:729] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc85883e044746a8a14cbf0aaf889c89, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.694929 9389 ts_tablet_manager.cc:1397] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:29.694964 9387 consensus_queue.cc:260] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.695066 9386 tablet_bootstrap.cc:654] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:29.695113 9387 ts_tablet_manager.cc:1428] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.695117 9389 raft_consensus.cc:348] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.695190 9389 raft_consensus.cc:374] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.695210 9389 raft_consensus.cc:729] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95c83cb078a4e5889ed5a9552238713, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.695250 9389 consensus_queue.cc:260] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.695361 9389 ts_tablet_manager.cc:1428] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:29.695500 9395 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.695467 9394 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.695578 9394 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.695593 9395 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.696010 9386 tablet_bootstrap.cc:492] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: No bootstrap required, opened a new log
I20260809 00:37:29.696089 9386 ts_tablet_manager.cc:1397] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:29.696239 9386 raft_consensus.cc:348] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.696342 9386 raft_consensus.cc:374] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:29.696359 9386 raft_consensus.cc:729] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e05fcc69ce5458f8eb45f2c703922a5, State: Initialized, Role: FOLLOWER
I20260809 00:37:29.696415 9386 consensus_queue.cc:260] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.696533 9386 ts_tablet_manager.cc:1428] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:29.696589 9392 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.696696 9392 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.860612 9394 raft_consensus.cc:482] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:29.860728 9394 raft_consensus.cc:504] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.860899 9394 leader_election.cc:288] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399), 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317)
I20260809 00:37:29.861161 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfbfe33344f4f48b9387ba16ebe7a30" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713" is_pre_election: true
I20260809 00:37:29.861240 9149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfbfe33344f4f48b9387ba16ebe7a30" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" is_pre_election: true
I20260809 00:37:29.861311 9004 raft_consensus.cc:2454] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 0.
I20260809 00:37:29.861361 9149 raft_consensus.cc:2454] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 0.
I20260809 00:37:29.861475 9233 leader_election.cc:302] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc85883e044746a8a14cbf0aaf889c89, f95c83cb078a4e5889ed5a9552238713; no voters:
I20260809 00:37:29.861609 9394 raft_consensus.cc:2781] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:29.861651 9394 raft_consensus.cc:482] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:29.861675 9394 raft_consensus.cc:3037] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.862418 9394 raft_consensus.cc:504] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.862551 9394 leader_election.cc:288] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 election: Requested vote from peers f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399), 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317)
I20260809 00:37:29.862751 9004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfbfe33344f4f48b9387ba16ebe7a30" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95c83cb078a4e5889ed5a9552238713"
I20260809 00:37:29.862780 9149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfbfe33344f4f48b9387ba16ebe7a30" candidate_uuid: "dc85883e044746a8a14cbf0aaf889c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e05fcc69ce5458f8eb45f2c703922a5"
I20260809 00:37:29.862838 9004 raft_consensus.cc:3037] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.862840 9149 raft_consensus.cc:3037] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:29.863699 9004 raft_consensus.cc:2454] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 1.
I20260809 00:37:29.863709 9149 raft_consensus.cc:2454] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc85883e044746a8a14cbf0aaf889c89 in term 1.
I20260809 00:37:29.863883 9233 leader_election.cc:302] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e05fcc69ce5458f8eb45f2c703922a5, dc85883e044746a8a14cbf0aaf889c89; no voters:
I20260809 00:37:29.863988 9394 raft_consensus.cc:2781] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:29.864045 9394 raft_consensus.cc:686] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [term 1 LEADER]: Becoming Leader. State: Replica: dc85883e044746a8a14cbf0aaf889c89, State: Running, Role: LEADER
I20260809 00:37:29.864133 9394 consensus_queue.cc:237] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } }
I20260809 00:37:29.864456 9397 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: TxnStatusTablet state changed. Reason: New leader dc85883e044746a8a14cbf0aaf889c89. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.864550 9397 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:29.864701 9433 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:37:29.864830 9433 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:37:29.864890 8834 catalog_manager.cc:5495] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } health_report { overall_health: HEALTHY } } }
I20260809 00:37:29.877851 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.875418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.7.105.67:43391)
0809 00:37:29.875467 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.875469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.875479 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:29.875573 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.875574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.875586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.875629 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:29.875760 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.876319 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.877097 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.877295 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.877370 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.877573 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.877611 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.877679 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.877751 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.877774 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:37:29.877871 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.875340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:43391 (local address 127.0.0.1:42852)
0809 00:37:29.875425 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.875438 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.875491 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.875648 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.875649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.875651 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:29.875699 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.875701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.876404 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.876405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.876998 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.877000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.877371 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.877372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.877442 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.877467 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.877682 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.877693 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.877746 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.877797 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:37:29.879037 9149 raft_consensus.cc:1261] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Refusing update from remote peer dc85883e044746a8a14cbf0aaf889c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.879043 9004 raft_consensus.cc:1261] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Refusing update from remote peer dc85883e044746a8a14cbf0aaf889c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.879267 9397 consensus_queue.cc:1035] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.879357 9394 consensus_queue.cc:1035] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.880021 9392 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: TxnStatusTablet state changed. Reason: New leader dc85883e044746a8a14cbf0aaf889c89. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.880221 9392 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.880477 9395 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: TxnStatusTablet state changed. Reason: New leader dc85883e044746a8a14cbf0aaf889c89. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.880575 9395 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.880995 9397 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.881085 9397 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:29.881515 9395 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.881609 9395 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P f95c83cb078a4e5889ed5a9552238713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.882493 9397 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.882579 9397 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P dc85883e044746a8a14cbf0aaf889c89: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:29.882745 9392 tablet_replica.cc:439] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc85883e044746a8a14cbf0aaf889c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 } } peers { permanent_uuid: "1e05fcc69ce5458f8eb45f2c703922a5" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34317 } } peers { permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 } } }
I20260809 00:37:29.882823 9392 tablet_replica.cc:442] T adfbfe33344f4f48b9387ba16ebe7a30 P 1e05fcc69ce5458f8eb45f2c703922a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:29.886022 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.883462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.7.105.66:34317)
0809 00:37:29.883500 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.883502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.883510 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:29.883547 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.883547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.883579 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.883660 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:29.883778 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.884365 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.885169 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.885377 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.885450 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.885635 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.885801 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.885883 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.885921 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.885946 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:29.891309 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.888706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:53519 (local address 127.7.105.67:43391)
0809 00:37:29.888755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.888757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.888875 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:37:29.888934 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.888935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.888946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.888993 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:29.889142 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.889571 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.890689 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.890860 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.890908 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.891106 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.891131 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.891168 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.891242 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.891258 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:29.891371 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.888622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:43391 (local address 127.7.105.66:53519)
0809 00:37:29.888801 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.888813 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.888875 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.889007 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.889008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.889010 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:29.889077 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.889079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.889639 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.889640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.890605 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.890608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.890929 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.890930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.891007 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.891025 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.891189 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.891191 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.891240 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.891283 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260809 00:37:29.893784 9004 raft_consensus.cc:1261] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Refusing update from remote peer 1e05fcc69ce5458f8eb45f2c703922a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.893844 9309 raft_consensus.cc:1261] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Refusing update from remote peer 1e05fcc69ce5458f8eb45f2c703922a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:29.894064 9419 consensus_queue.cc:1035] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f95c83cb078a4e5889ed5a9552238713" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.894208 9392 consensus_queue.cc:1035] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc85883e044746a8a14cbf0aaf889c89" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:29.896235 9401 mvcc.cc:205] Tried to move back new op lower bound from 7316422041163284480 to 7316422040196390912. Current Snapshot: MvccSnapshot[applied={T|T < 7316422041163284480}]
I20260809 00:37:29.904093 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.900978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:38469 (local address 127.7.105.66:34317)
0809 00:37:29.901020 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.901023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.901238 (+ 215us) server_negotiation.cc:407] Connection header received
0809 00:37:29.901271 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.901272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.901285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.901345 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:29.901456 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.902064 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.903254 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.903553 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.903621 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.903813 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.903846 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.903925 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.903982 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.904001 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:29.904138 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.900937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:34317 (local address 127.7.105.67:38469)
0809 00:37:29.901156 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:29.901168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:29.901227 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:29.901349 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:29.901350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:29.901351 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:29.901399 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.901401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.902141 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.902142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.903102 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:29.903105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.903620 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.903621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:29.903685 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.903726 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:29.903928 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:29.903931 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:29.903971 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 00:37:29.904018 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":139,"threads_started":1}
I20260809 00:37:29.914044 8835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:29.914170 8835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:29.914835 8835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da54b8f5d1684afc865136f34f9ac76c: Sending DeleteTablet for 3 replicas of tablet ab8a07cabd8a462082e57d51e206297c
I20260809 00:37:29.915145 8984 tablet_service.cc:1514] Processing DeleteTablet for tablet ab8a07cabd8a462082e57d51e206297c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46154
I20260809 00:37:29.915167 9132 tablet_service.cc:1514] Processing DeleteTablet for tablet ab8a07cabd8a462082e57d51e206297c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:34224
I20260809 00:37:29.915246 9411 tablet_replica.cc:330] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: stopping tablet replica
I20260809 00:37:29.915252 9412 tablet_replica.cc:330] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: stopping tablet replica
I20260809 00:37:29.915294 9411 raft_consensus.cc:2229] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:29.915295 9412 raft_consensus.cc:2229] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:29.915329 9411 raft_consensus.cc:2258] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.915374 9412 raft_consensus.cc:2258] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.915517 9286 tablet_service.cc:1514] Processing DeleteTablet for tablet ab8a07cabd8a462082e57d51e206297c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:42832
I20260809 00:37:29.915643 9413 tablet_replica.cc:330] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: stopping tablet replica
I20260809 00:37:29.915704 9413 raft_consensus.cc:2229] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:29.915750 9413 raft_consensus.cc:2258] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:29.916108 9413 ts_tablet_manager.cc:1905] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.916114 9411 ts_tablet_manager.cc:1905] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.917870 9412 ts_tablet_manager.cc:1905] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:29.917908 9413 ts_tablet_manager.cc:1918] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:29.917920 9411 ts_tablet_manager.cc:1918] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:29.917958 9413 log.cc:1196] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/ab8a07cabd8a462082e57d51e206297c
I20260809 00:37:29.917979 9411 log.cc:1196] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/ab8a07cabd8a462082e57d51e206297c
I20260809 00:37:29.918213 9411 ts_tablet_manager.cc:1939] T ab8a07cabd8a462082e57d51e206297c P f95c83cb078a4e5889ed5a9552238713: Deleting consensus metadata
I20260809 00:37:29.918212 9413 ts_tablet_manager.cc:1939] T ab8a07cabd8a462082e57d51e206297c P dc85883e044746a8a14cbf0aaf889c89: Deleting consensus metadata
I20260809 00:37:29.918552 8819 catalog_manager.cc:4841] TS f95c83cb078a4e5889ed5a9552238713 (127.7.105.65:43399): tablet ab8a07cabd8a462082e57d51e206297c (table test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]) successfully deleted
I20260809 00:37:29.919471 9412 ts_tablet_manager.cc:1918] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:29.919523 9412 log.cc:1196] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/ab8a07cabd8a462082e57d51e206297c
I20260809 00:37:29.919742 9412 ts_tablet_manager.cc:1939] T ab8a07cabd8a462082e57d51e206297c P 1e05fcc69ce5458f8eb45f2c703922a5: Deleting consensus metadata
I20260809 00:37:29.920308 8819 catalog_manager.cc:4841] TS 1e05fcc69ce5458f8eb45f2c703922a5 (127.7.105.66:34317): tablet ab8a07cabd8a462082e57d51e206297c (table test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]) successfully deleted
I20260809 00:37:29.920825 8821 catalog_manager.cc:4841] TS dc85883e044746a8a14cbf0aaf889c89 (127.7.105.67:43391): tablet ab8a07cabd8a462082e57d51e206297c (table test-table [id=ccc5b35da2b84f9d830c81f18aa73cde]) successfully deleted
Aug 09 00:37:29 dist-test-slave-1zqn krb5kdc[8787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235849, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:29.931941 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.920927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.7.105.65:43399)
0809 00:37:29.920969 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.920971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.921272 (+ 301us) server_negotiation.cc:407] Connection header received
0809 00:37:29.921322 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.921323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.921338 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.921407 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.922097 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.922849 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.923943 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.924273 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.924354 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.928174 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.928189 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.928190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.928199 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.930350 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.930859 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.930864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.930864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.930939 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.931170 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.931171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.931171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.931367 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.931624 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.931669 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.931801 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:37:29.932792 9045 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162
I20260809 00:37:29.942055 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:29.933936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.7.105.126:36329)
0809 00:37:29.933983 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:29.933985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:29.934241 (+ 256us) server_negotiation.cc:407] Connection header received
0809 00:37:29.934300 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:29.934301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:29.934314 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:29.934375 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:29.935059 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.935859 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.936919 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:29.937237 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:29.937320 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:29.938883 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:29.938884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:29.938885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:29.938893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:29.940768 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.941198 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.941200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.941200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.941261 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:29.941485 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:29.941485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:29.941486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:29.941629 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:29.941730 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:29.941875 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:37:29.941956 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:37:29.942391 8835 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58682
I20260809 00:37:29.943133 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8912
I20260809 00:37:29.962585 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9062
I20260809 00:37:29.970077 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9215
I20260809 00:37:29.979198 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 8801
2026-08-09T00:37:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2663 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:30 dist-test-slave-1zqn krb5kdc[9457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:30 dist-test-slave-1zqn krb5kdc[9457](info): set up 2 sockets
Aug 09 00:37:30 dist-test-slave-1zqn krb5kdc[9457](info): commencing operation
krb5kdc: starting...
W20260809 00:37:31.165261 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:31.295261 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44887
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:31.379319 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:31.379582 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:31.379624 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:31.380838 9472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:31.380916 9472 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:37:31.380939 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:31.380959 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:31.381017 9472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:31.381047 9472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:31.382570 9472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44887
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:31.382874 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:31.383123 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:31.388513 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.388513 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:31.388810 9472 server_base.cc:1034] running on GCE node
W20260809 00:37:31.388556 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:31.389231 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:31.389554 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:31.390735 9472 hybrid_clock.cc:648] HybridClock initialized: now 1786235851390720 us; error 47 us; skew 500 ppm
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:31.393458 9472 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:31.393966 9472 webserver.cc:460] Webserver started at http://127.7.105.126:35137/ using document root <none> and password file <none>
I20260809 00:37:31.394258 9472 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:31.394332 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:31.394487 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:31.395584 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "6546377d0f6a442c80c71772002df19d"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "dacc7e3648f1ee2fa2ad9b2099483092"
server_key_iv: "b2a163f7fd8f1bf4761d7446b22404a8"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.395952 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "6546377d0f6a442c80c71772002df19d"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "dacc7e3648f1ee2fa2ad9b2099483092"
server_key_iv: "b2a163f7fd8f1bf4761d7446b22404a8"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.397346 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:31.398099 9488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.398275 9472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:31.398386 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "6546377d0f6a442c80c71772002df19d"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "dacc7e3648f1ee2fa2ad9b2099483092"
server_key_iv: "b2a163f7fd8f1bf4761d7446b22404a8"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.398463 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:31.413239 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:31.415155 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:31.415328 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:31.420358 9472 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44887
I20260809 00:37:31.420413 9557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44887 every 8 connection(s)
I20260809 00:37:31.420782 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:31.421509 9558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:31.424160 9558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Bootstrap starting.
I20260809 00:37:31.424775 9558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:31.425036 9558 log.cc:824] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:31.425719 9558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: No bootstrap required, opened a new log
I20260809 00:37:31.427946 9558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } }
I20260809 00:37:31.428164 9558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:31.428210 9558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546377d0f6a442c80c71772002df19d, State: Initialized, Role: FOLLOWER
I20260809 00:37:31.428328 9558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } }
I20260809 00:37:31.428390 9558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:31.428413 9558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:31.428449 9558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:31.428984 9558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } }
I20260809 00:37:31.429078 9558 leader_election.cc:302] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6546377d0f6a442c80c71772002df19d; no voters:
I20260809 00:37:31.429212 9558 leader_election.cc:288] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:31.429276 9561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:31.429445 9561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [term 1 LEADER]: Becoming Leader. State: Replica: 6546377d0f6a442c80c71772002df19d, State: Running, Role: LEADER
I20260809 00:37:31.429481 9558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:31.429545 9561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } }
I20260809 00:37:31.429960 9562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6546377d0f6a442c80c71772002df19d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } } }
I20260809 00:37:31.430090 9562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:31.429976 9563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6546377d0f6a442c80c71772002df19d. Latest consensus state: current_term: 1 leader_uuid: "6546377d0f6a442c80c71772002df19d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6546377d0f6a442c80c71772002df19d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44887 } } }
I20260809 00:37:31.430270 9563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:31.430418 9568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:31.430963 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9472
I20260809 00:37:31.430984 9568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:31.431077 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:31.431299 7589 external_mini_cluster.cc:1428] Setting key f0e6541c62dbc4058887b10ab3621ab8
I20260809 00:37:31.432753 9568 catalog_manager.cc:1347] Generated new cluster ID: b1bb8c42801b4728875a5420e76caab7
I20260809 00:37:31.432809 9568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:31.442122 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.431841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.7.105.126:44887)
0809 00:37:31.432082 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.432092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.432128 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:37:31.432253 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.432260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.432325 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.432482 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.433293 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.434321 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.435142 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.435629 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.435790 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.438338 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.438342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.438343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.438356 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.440328 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.440764 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.440768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.440768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.440885 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.441130 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.441131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.441132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.441388 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.441584 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.441663 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.441928 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260809 00:37:31.452903 9568 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:31.453691 9568 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:31.459374 9568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Generated new TSK 0
I20260809 00:37:31.459612 9568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:31.498478 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44887
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:31.610615 9584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:31.610859 9584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:31.610888 9584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:31.612094 9584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:31.612133 9584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:31.612154 9584 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:37:31.612165 9584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:31.612183 9584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:31.613690 9584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:31.613898 9584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:31.614090 9584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:31.619412 9590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.619424 9591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.619515 9593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:31.619728 9584 server_base.cc:1034] running on GCE node
I20260809 00:37:31.619905 9584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:31.620169 9584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:31.621311 9584 hybrid_clock.cc:648] HybridClock initialized: now 1786235851621298 us; error 29 us; skew 500 ppm
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:31.623808 9584 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:31.624297 9584 webserver.cc:460] Webserver started at http://127.7.105.65:35905/ using document root <none> and password file <none>
I20260809 00:37:31.624591 9584 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:31.624666 9584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:31.624805 9584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:31.625869 9584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "dabb54e2bcbd47108148358285282809"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "c14000328d5fa5d149e23c8ded9ef121"
server_key_iv: "f52aee85606c9cc55abb41ffd32c5a13"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.626235 9584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "dabb54e2bcbd47108148358285282809"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "c14000328d5fa5d149e23c8ded9ef121"
server_key_iv: "f52aee85606c9cc55abb41ffd32c5a13"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.627588 9584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:31.628376 9600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.628571 9584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:31.628708 9584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "dabb54e2bcbd47108148358285282809"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "c14000328d5fa5d149e23c8ded9ef121"
server_key_iv: "f52aee85606c9cc55abb41ffd32c5a13"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.628814 9584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:31.644430 9584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:31.646343 9584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:31.646525 9584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:31.647081 9584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:31.647145 9584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.647192 9584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:31.647215 9584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:31.654440 9584 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:42153
I20260809 00:37:31.654531 9726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:42153 every 8 connection(s)
I20260809 00:37:31.654835 9584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:31.655159 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9584
I20260809 00:37:31.655269 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:31.655494 7589 external_mini_cluster.cc:1428] Setting key eb6a2a18a7758ffb63c816a7c7b4db0b
I20260809 00:37:31.658291 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.647656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53879 (local address 127.7.105.126:44887)
0809 00:37:31.647733 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.647737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.647979 (+ 242us) server_negotiation.cc:407] Connection header received
0809 00:37:31.648539 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.648540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.648557 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.648631 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.649491 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.650304 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.652001 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.652407 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.652487 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.654949 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.654951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.654951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.654962 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.656881 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.657283 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.657284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.657284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.657350 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.657579 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.657580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.657580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.657730 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.657799 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.658090 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.658197 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:37:31.658526 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.647634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.65:53879)
0809 00:37:31.647822 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.647864 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.648430 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.648698 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.648702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.649086 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.649401 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.649409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.650414 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.650416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.651908 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.651912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.652498 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.652499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.652658 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.653191 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.653201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.654843 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.656974 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.656981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.656982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.657209 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.657417 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.657417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.657418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.657515 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.657794 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.657799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.657909 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.658073 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.658268 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260809 00:37:31.659118 9727 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44887
I20260809 00:37:31.659229 9727 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:31.659433 9727 heartbeater.cc:511] Master 127.7.105.126:44887 requested a full tablet report, sending...
I20260809 00:37:31.660029 9505 ts_manager.cc:194] Registered new tserver with Master: dabb54e2bcbd47108148358285282809 (127.7.105.65:42153)
I20260809 00:37:31.661110 9505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:53879
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:37:31.666806 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.660234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.65:55993)
0809 00:37:31.660382 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.660393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.660469 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.660832 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.660833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.661125 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.661280 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.661283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.662117 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.662118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.662632 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.662633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.663002 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.663003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.663084 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.663553 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.663558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.663835 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.665536 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.665537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.665537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.665709 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.665962 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.665963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.665963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.666027 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.666537 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.666537 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.666594 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.666656 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.666739 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260809 00:37:31.666826 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.660228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:55993 (local address 127.7.105.126:44887)
0809 00:37:31.660268 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.660269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.660445 (+ 176us) server_negotiation.cc:407] Connection header received
0809 00:37:31.660689 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.660690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.660703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.660736 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.661351 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.662034 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.662718 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.662929 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.663003 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.663949 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.663950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.663950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.663957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.665433 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.665799 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.665800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.665800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.665855 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.666116 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.666117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.666118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.666405 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.666532 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.666690 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.666761 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:31.687201 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44887
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:31.774271 9736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:31.774502 9736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:31.774529 9736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:31.775728 9736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:31.775774 9736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:31.775795 9736 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:37:31.775844 9736 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:31.775871 9736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:31.777426 9736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:31.777681 9736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:31.777921 9736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:31.783345 9741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.783362 9745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.783362 9743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:31.783540 9736 server_base.cc:1034] running on GCE node
I20260809 00:37:31.783744 9736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:31.784003 9736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:31.785202 9736 hybrid_clock.cc:648] HybridClock initialized: now 1786235851785187 us; error 32 us; skew 500 ppm
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:31.787824 9736 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:31.788349 9736 webserver.cc:460] Webserver started at http://127.7.105.66:45119/ using document root <none> and password file <none>
I20260809 00:37:31.788832 9736 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:31.789044 9736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:31.789235 9736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:31.790472 9736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "3ed80988d52c429192ff22c07b6e440b"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "08c0ea992d6f2a65008b9609c0e4a2ca"
server_key_iv: "70365bf348845f52342a9936143dee81"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.790869 9736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "3ed80988d52c429192ff22c07b6e440b"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "08c0ea992d6f2a65008b9609c0e4a2ca"
server_key_iv: "70365bf348845f52342a9936143dee81"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.792424 9736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 00:37:31.793468 9753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.793689 9736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:31.793799 9736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "3ed80988d52c429192ff22c07b6e440b"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "08c0ea992d6f2a65008b9609c0e4a2ca"
server_key_iv: "70365bf348845f52342a9936143dee81"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.793879 9736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:31.817474 9736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:31.819587 9736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:31.819758 9736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:31.820478 9736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:31.820533 9736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.820564 9736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:31.820583 9736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:31.828028 9736 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:43363
I20260809 00:37:31.828172 9886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:43363 every 8 connection(s)
I20260809 00:37:31.828528 9736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:31.831519 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.820906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:48749 (local address 127.7.105.126:44887)
0809 00:37:31.820984 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.820986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.821227 (+ 241us) server_negotiation.cc:407] Connection header received
0809 00:37:31.821758 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.821760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.821775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.821858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.822890 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.823736 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.825066 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.825428 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.825513 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.828452 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.828453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.828454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.828463 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.830017 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.830536 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.830536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.830537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.830599 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.830863 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.830863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.830863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.830972 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.831024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.831327 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.831397 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:37:31.831647 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.820896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.66:48749)
0809 00:37:31.821087 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.821130 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.821657 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.822012 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.822016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.822444 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.822751 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.822759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.823866 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.823867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.824962 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.824965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.825538 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.825539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.825680 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.826266 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.826278 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.828346 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.830124 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.830131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.830133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.830479 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.830685 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.830686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.830686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.830797 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.831037 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.831041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.831159 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.831292 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.831436 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":150,"threads_started":1}
I20260809 00:37:31.832250 9887 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44887
I20260809 00:37:31.832331 9887 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:31.832561 9887 heartbeater.cc:511] Master 127.7.105.126:44887 requested a full tablet report, sending...
I20260809 00:37:31.833001 9506 ts_manager.cc:194] Registered new tserver with Master: 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363)
I20260809 00:37:31.833423 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9736
I20260809 00:37:31.833503 9506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:48749
I20260809 00:37:31.833539 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:31.833729 7589 external_mini_cluster.cc:1428] Setting key 22eac0b30745004f2aa1bc23eace88e0
I20260809 00:37:31.839578 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.833363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.66:59319)
0809 00:37:31.833507 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.833520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.833608 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.833753 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.833754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.833956 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.834021 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.834023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.834885 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.834885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.835414 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.835416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.835794 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.835795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.835861 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.836359 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.836364 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.836658 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.838488 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.838489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.838489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.838674 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.838900 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.838900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.838901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.838952 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.839310 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.839310 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.839348 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.839407 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.839483 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 00:37:31.839648 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.833407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:59319 (local address 127.7.105.126:44887)
0809 00:37:31.833447 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.833449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.833583 (+ 134us) server_negotiation.cc:407] Connection header received
0809 00:37:31.833671 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.833672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.833681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.833734 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.834103 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.834784 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.835518 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.835719 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.835795 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.836748 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.836749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.836749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.836758 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.838403 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.838754 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.838756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.838756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.838828 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.839014 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.839016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.839016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.839234 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.839314 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.839457 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.839536 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:31.862866 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44887
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:31.948311 9896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:31.948523 9896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:31.948551 9896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:31.949805 9896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:31.949855 9896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:31.949874 9896 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:37:31.949927 9896 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:31.949957 9896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:31.951393 9896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:31.951629 9896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:31.951857 9896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:31.957022 9901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.957108 9902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:31.957123 9904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:31.957449 9896 server_base.cc:1034] running on GCE node
I20260809 00:37:31.957690 9896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:31.957958 9896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:31.959106 9896 hybrid_clock.cc:648] HybridClock initialized: now 1786235851959082 us; error 44 us; skew 500 ppm
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:31.961827 9896 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:31.962198 9896 webserver.cc:460] Webserver started at http://127.7.105.67:37093/ using document root <none> and password file <none>
I20260809 00:37:31.962404 9896 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:31.962445 9896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:31.962560 9896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:31.963539 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "f203d3dd92abd887f84632c10e819d63"
server_key_iv: "1906f4ef13a864215558dd2d257b2542"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.963828 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "f203d3dd92abd887f84632c10e819d63"
server_key_iv: "1906f4ef13a864215558dd2d257b2542"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.965286 9896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:31.965973 9911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.966180 9896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:31.966287 9896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
format_stamp: "Formatted at 2026-08-09 00:37:31 on dist-test-slave-1zqn"
server_key: "f203d3dd92abd887f84632c10e819d63"
server_key_iv: "1906f4ef13a864215558dd2d257b2542"
server_key_version: "encryptionkey@0"
I20260809 00:37:31.966363 9896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:31.973889 9896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:31.975899 9896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:31.976109 9896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:31.976608 9896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:31.976651 9896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:31.976675 9896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:31.976722 9896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:31.984356 9896 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:41517
I20260809 00:37:31.984481 10039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:41517 every 8 connection(s)
I20260809 00:37:31.984786 9896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:31.987535 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.977132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:59073 (local address 127.7.105.126:44887)
0809 00:37:31.977176 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.977179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.977387 (+ 208us) server_negotiation.cc:407] Connection header received
0809 00:37:31.977932 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.977934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.977948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.978007 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.978968 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.979801 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.981498 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.981821 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.981901 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.984666 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.984669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.984670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.984682 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.986220 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.986609 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.986610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.986610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.986675 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.986927 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.986928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.986929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.987102 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.987166 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.987398 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.987467 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:31.987692 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.977117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.67:59073)
0809 00:37:31.977271 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.977305 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.977810 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.978131 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.978135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.978532 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.978853 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.978861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.979904 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.979905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.981401 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.981405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.981938 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.981939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.982113 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.982697 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.982708 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.984562 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.986295 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.986300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.986302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.986516 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.986746 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.986746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.986747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.986845 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.987168 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.987173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.987285 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.987367 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.987490 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260809 00:37:31.988216 10040 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44887
I20260809 00:37:31.988320 10040 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:31.988497 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 9896
I20260809 00:37:31.988587 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:31.988559 10040 heartbeater.cc:511] Master 127.7.105.126:44887 requested a full tablet report, sending...
I20260809 00:37:31.988778 7589 external_mini_cluster.cc:1428] Setting key d829f9f7b881f2add26c18eb24abb749
I20260809 00:37:31.989180 9506 ts_manager.cc:194] Registered new tserver with Master: e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:31.989691 9506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:59073
I20260809 00:37:31.990403 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:31.995643 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.989352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:58441 (local address 127.7.105.126:44887)
0809 00:37:31.989921 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:31.989923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:31.989938 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:37:31.989955 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:31.989956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:31.989963 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:31.990053 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:31.990440 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.990937 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.991638 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.991847 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.991922 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.993118 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:31.993119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:31.993120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:31.993126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:31.994505 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.994857 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.994858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.994859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.994914 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:31.995123 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:31.995124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:31.995124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:31.995236 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:31.995309 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:31.995471 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:37:31.995548 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550}
I20260809 00:37:31.995661 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:31.989321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.7.105.67:58441)
0809 00:37:31.989467 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:31.989478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:31.989533 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:31.990072 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:31.990073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:31.990279 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:31.990365 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.990368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.991040 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.991041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.991544 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:31.991546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:31.991921 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:31.991922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:31.991998 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:31.992512 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:31.992519 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:31.993016 (+ 497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:31.994612 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.994613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.994614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.994783 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.994989 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:31.994990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:31.994990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:31.995051 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:31.995300 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:31.995301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:31.995358 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:37:31.995444 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:37:31.995558 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
Aug 09 00:37:31 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:37:32 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:32.015764 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.008579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.7.105.126:44887)
0809 00:37:32.008646 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.008648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.008658 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:32.008743 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.008744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.008766 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.008832 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:32.009437 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.009933 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.010688 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.010885 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.010985 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.013274 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:32.013275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:32.013276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:32.013284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:32.014608 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:32.014966 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:32.014967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:32.014967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:32.015028 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:32.015243 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:32.015244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:32.015245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:32.015357 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:32.015465 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.015562 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.015654 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:32.017367 9506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:32.018985 9506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:32.025723 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:41517 (local address 127.0.0.1:46282)
0809 00:37:32.022250 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.022261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.022312 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.022677 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.022678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.022679 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:32.022714 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.022715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.023523 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.023524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.024510 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.024513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.025423 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.025424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.025506 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.025521 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.025598 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.025648 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260809 00:37:32.025887 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42153 (local address 127.0.0.1:46324)
0809 00:37:32.022013 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.022024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.022108 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.022387 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.022388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.022394 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:32.022481 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.022484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.023378 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.023379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.024688 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.024690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.025626 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.025628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.025691 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.025696 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.025754 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.025795 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":51,"thread_start_us":81,"threads_started":1}
I20260809 00:37:32.026037 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.7.105.67:41517)
0809 00:37:32.022251 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.022256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.022547 (+ 291us) server_negotiation.cc:407] Connection header received
0809 00:37:32.022585 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.022586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.022598 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.022679 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:32.022770 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.023436 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.024596 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.025326 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.025407 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.025465 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.025905 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.025948 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":139,"threads_started":1}
I20260809 00:37:32.026590 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.0.0.1:56504)
0809 00:37:32.021545 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.021592 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.021665 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.022385 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.022388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.022390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:32.022481 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.022483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.024621 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.024622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.025410 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.025413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.026337 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.026338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.026410 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.026414 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.026471 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.026510 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":5,"thread_start_us":53,"threads_started":1}
I20260809 00:37:32.026613 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.7.105.66:43363)
0809 00:37:32.021981 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.021985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.021995 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:32.022029 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.022048 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.022065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.022152 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:32.023412 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.024526 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.025506 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.026247 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.026335 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.026392 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.026484 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.026520 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":104,"threads_started":1}
I20260809 00:37:32.026751 9967 tablet_service.cc:1467] Processing CreateTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (DEFAULT_TABLE table=test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.027120 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b54fa0d597d94b1d91e22c7ccb2b7a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.027237 9809 tablet_service.cc:1467] Processing CreateTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (DEFAULT_TABLE table=test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.027582 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b54fa0d597d94b1d91e22c7ccb2b7a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.030213 10066 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Bootstrap starting.
I20260809 00:37:32.030586 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.021751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.7.105.65:42153)
0809 00:37:32.021953 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.021959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.022079 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:37:32.022201 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.022204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.022221 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.022278 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:32.022559 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.023286 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.024787 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.025541 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.025625 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.025683 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.025769 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.030453 (+ 4684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260809 00:37:32.030948 10067 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Bootstrap starting.
I20260809 00:37:32.031262 9653 tablet_service.cc:1467] Processing CreateTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (DEFAULT_TABLE table=test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.031452 10066 tablet_bootstrap.cc:654] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.031599 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b54fa0d597d94b1d91e22c7ccb2b7a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.031644 10067 tablet_bootstrap.cc:654] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.031806 10066 log.cc:824] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:32.032035 10067 log.cc:824] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:32.032634 10066 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: No bootstrap required, opened a new log
I20260809 00:37:32.032889 10066 ts_tablet_manager.cc:1397] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:37:32.032954 10067 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: No bootstrap required, opened a new log
I20260809 00:37:32.033035 10067 ts_tablet_manager.cc:1397] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:37:32.034508 10067 raft_consensus.cc:348] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.034644 10067 raft_consensus.cc:374] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.034667 10067 raft_consensus.cc:729] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.034792 10067 consensus_queue.cc:260] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.035097 10067 ts_tablet_manager.cc:1428] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:37:32.035254 9887 heartbeater.cc:503] Master 127.7.105.126:44887 was elected leader, sending a full tablet report...
I20260809 00:37:32.035374 10066 raft_consensus.cc:348] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.035553 10066 raft_consensus.cc:374] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.035609 10066 raft_consensus.cc:729] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.035663 10071 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Bootstrap starting.
I20260809 00:37:32.035722 10066 consensus_queue.cc:260] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.036005 10040 heartbeater.cc:503] Master 127.7.105.126:44887 was elected leader, sending a full tablet report...
I20260809 00:37:32.036245 10066 ts_tablet_manager.cc:1428] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:32.036374 10071 tablet_bootstrap.cc:654] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.036731 10071 log.cc:824] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:32.037492 10071 tablet_bootstrap.cc:492] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: No bootstrap required, opened a new log
I20260809 00:37:32.037549 10071 ts_tablet_manager.cc:1397] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:32.038831 10071 raft_consensus.cc:348] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.038949 10071 raft_consensus.cc:374] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.038969 10071 raft_consensus.cc:729] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb54e2bcbd47108148358285282809, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.039049 10071 consensus_queue.cc:260] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.039229 10071 ts_tablet_manager.cc:1428] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:32.039314 9727 heartbeater.cc:503] Master 127.7.105.126:44887 was elected leader, sending a full tablet report...
W20260809 00:37:32.079200 9888 tablet.cc:2369] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:37:32.155900 9728 tablet.cc:2369] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:37:32.235477 10041 tablet.cc:2369] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:32.328868 10070 raft_consensus.cc:482] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:32.329006 10070 raft_consensus.cc:504] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.329378 10070 leader_election.cc:288] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.332628 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.329641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42153 (local address 127.7.105.66:41999)
0809 00:37:32.329723 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.329740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.329854 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.329983 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.329984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.329985 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:32.330059 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.330066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.330864 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.330865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.331734 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.331755 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.332356 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.332357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.332446 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.332449 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.332505 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.332550 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":2}
I20260809 00:37:32.332628 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.329694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41999 (local address 127.7.105.65:42153)
0809 00:37:32.329740 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.329742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.329833 (+ 91us) server_negotiation.cc:407] Connection header received
0809 00:37:32.329919 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.329920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.329930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.329981 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:32.330129 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.330786 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.331824 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.332272 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.332349 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.332358 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.332523 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.332562 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:32.332814 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.329880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:55863 (local address 127.7.105.67:41517)
0809 00:37:32.329928 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.329930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.329939 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:32.330027 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.330028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.330037 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.330130 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:32.330284 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.330868 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.332033 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.332449 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.332534 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.332542 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.332687 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.332721 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:37:32.332825 10077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.329757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:41517 (local address 127.7.105.66:55863)
0809 00:37:32.329887 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.329898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.329951 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.330134 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.330135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.330137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:32.330216 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.330218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.330958 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.330959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.331930 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.331933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.332535 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.332536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.332627 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.332631 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.332682 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.332720 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":28,"thread_start_us":64,"threads_started":1}
I20260809 00:37:32.332993 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54fa0d597d94b1d91e22c7ccb2b7a67" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809" is_pre_election: true
I20260809 00:37:32.333165 9676 raft_consensus.cc:2454] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.333400 9756 leader_election.cc:302] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, dabb54e2bcbd47108148358285282809; no voters:
I20260809 00:37:32.333577 10070 raft_consensus.cc:2781] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:32.333539 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54fa0d597d94b1d91e22c7ccb2b7a67" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" is_pre_election: true
I20260809 00:37:32.333629 10070 raft_consensus.cc:482] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:32.333652 10070 raft_consensus.cc:3037] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.333689 9989 raft_consensus.cc:2454] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.334442 10070 raft_consensus.cc:504] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.334581 10070 leader_election.cc:288] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Requested vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.334738 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54fa0d597d94b1d91e22c7ccb2b7a67" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809"
I20260809 00:37:32.334748 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54fa0d597d94b1d91e22c7ccb2b7a67" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
I20260809 00:37:32.334820 9989 raft_consensus.cc:3037] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.334820 9676 raft_consensus.cc:3037] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.335676 9989 raft_consensus.cc:2454] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.335676 9676 raft_consensus.cc:2454] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.335863 9756 leader_election.cc:302] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:32.335950 10070 raft_consensus.cc:2781] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:32.336138 10070 raft_consensus.cc:686] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 1 LEADER]: Becoming Leader. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Running, Role: LEADER
I20260809 00:37:32.336236 10070 consensus_queue.cc:237] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.337047 9506 catalog_manager.cc:5495] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed80988d52c429192ff22c07b6e440b (127.7.105.66). New cstate: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:37:32 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:32.395452 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.388830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44887 (local address 127.0.0.1:52428)
0809 00:37:32.388995 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.389004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.389098 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.389238 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.389239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.389563 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:32.389614 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.389616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.390264 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.390265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.390796 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.390798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.391124 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.391124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.391169 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.391658 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:32.391662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:32.393188 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:32.394531 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:32.394538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:32.394541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:32.394749 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:32.394916 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:32.394916 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:32.394917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:32.394972 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:32.395166 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:32.395172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:32.395211 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.395267 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.395358 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260809 00:37:32.395501 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.388870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.7.105.126:44887)
0809 00:37:32.388925 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.388928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.389070 (+ 142us) server_negotiation.cc:407] Connection header received
0809 00:37:32.389155 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.389156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.389171 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.389222 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:32.389689 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.390184 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.390870 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.391057 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.391113 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.393266 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:32.393267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:32.393268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:32.393274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:32.394463 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:32.394812 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:32.394812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:32.394813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:32.394862 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:32.395020 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:32.395021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:32.395021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:32.395117 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:32.395165 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.395297 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.395404 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":28}
I20260809 00:37:32.397782 9506 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:52428:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:37:32.397989 9506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:32.399490 9653 tablet_service.cc:1467] Processing CreateTablet for tablet 062a4adb716f450a9ee2da1f89b5304a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79ff9dfabaaa4a23a0f4a9d00b921715]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:32.399513 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 062a4adb716f450a9ee2da1f89b5304a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79ff9dfabaaa4a23a0f4a9d00b921715]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:32.399564 9967 tablet_service.cc:1467] Processing CreateTablet for tablet 062a4adb716f450a9ee2da1f89b5304a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79ff9dfabaaa4a23a0f4a9d00b921715]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:32.399688 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062a4adb716f450a9ee2da1f89b5304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.399690 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062a4adb716f450a9ee2da1f89b5304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.399698 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062a4adb716f450a9ee2da1f89b5304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.400949 10071 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: Bootstrap starting.
I20260809 00:37:32.401260 10066 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: Bootstrap starting.
I20260809 00:37:32.401304 10067 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: Bootstrap starting.
I20260809 00:37:32.401404 10071 tablet_bootstrap.cc:654] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.401665 10067 tablet_bootstrap.cc:654] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.401837 10066 tablet_bootstrap.cc:654] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.402185 10071 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: No bootstrap required, opened a new log
I20260809 00:37:32.402245 10071 ts_tablet_manager.cc:1397] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:32.402423 10071 raft_consensus.cc:348] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.402513 10071 raft_consensus.cc:374] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.402554 10071 raft_consensus.cc:729] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb54e2bcbd47108148358285282809, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.402597 10071 consensus_queue.cc:260] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.402658 10066 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: No bootstrap required, opened a new log
I20260809 00:37:32.402717 10066 ts_tablet_manager.cc:1397] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:32.402715 10071 ts_tablet_manager.cc:1428] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:32.402781 10076 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.402875 10076 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.402863 10066 raft_consensus.cc:348] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.402947 10066 raft_consensus.cc:374] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.402971 10066 raft_consensus.cc:729] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.403010 10066 consensus_queue.cc:260] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.403139 10066 ts_tablet_manager.cc:1428] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:32.403261 10067 tablet_bootstrap.cc:492] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: No bootstrap required, opened a new log
I20260809 00:37:32.403210 10072 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.403306 10067 ts_tablet_manager.cc:1397] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:37:32.403303 10072 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.403450 10067 raft_consensus.cc:348] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.403519 10067 raft_consensus.cc:374] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.403537 10067 raft_consensus.cc:729] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.403590 10067 consensus_queue.cc:260] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.403681 10067 ts_tablet_manager.cc:1428] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:32.403733 10070 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.403846 10070 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.571113 10070 raft_consensus.cc:482] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:32.571239 10070 raft_consensus.cc:504] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.571436 10070 leader_election.cc:288] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.571727 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062a4adb716f450a9ee2da1f89b5304a" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809" is_pre_election: true
I20260809 00:37:32.571756 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062a4adb716f450a9ee2da1f89b5304a" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" is_pre_election: true
I20260809 00:37:32.571849 9676 raft_consensus.cc:2454] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.571849 9989 raft_consensus.cc:2454] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.572006 9756 leader_election.cc:302] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:32.572147 10070 raft_consensus.cc:2781] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:32.572202 10070 raft_consensus.cc:482] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:32.572232 10070 raft_consensus.cc:3037] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.572958 10070 raft_consensus.cc:504] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.573094 10070 leader_election.cc:288] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Requested vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.573310 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062a4adb716f450a9ee2da1f89b5304a" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
I20260809 00:37:32.573312 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062a4adb716f450a9ee2da1f89b5304a" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809"
I20260809 00:37:32.573402 9989 raft_consensus.cc:3037] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.573402 9676 raft_consensus.cc:3037] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.574191 9676 raft_consensus.cc:2454] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.574191 9989 raft_consensus.cc:2454] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.574373 9756 leader_election.cc:302] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, dabb54e2bcbd47108148358285282809; no voters:
I20260809 00:37:32.574476 10070 raft_consensus.cc:2781] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:32.574575 10070 raft_consensus.cc:686] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [term 1 LEADER]: Becoming Leader. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Running, Role: LEADER
I20260809 00:37:32.574636 10070 consensus_queue.cc:237] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.574940 10078 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: TxnStatusTablet state changed. Reason: New leader 3ed80988d52c429192ff22c07b6e440b. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.575079 10078 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:32.575250 10091 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:37:32.575383 10091 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:37:32.575423 9506 catalog_manager.cc:5495] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed80988d52c429192ff22c07b6e440b (127.7.105.66). New cstate: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:32.588117 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.585321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.7.105.66:43363)
0809 00:37:32.585500 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.585502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.585515 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:37:32.585549 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.585549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.585569 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.585654 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:32.585818 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.586377 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.587184 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.587433 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.587519 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.587719 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:32.587839 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:32.587924 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.587966 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.587988 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260809 00:37:32.588135 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.585195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.0.0.1:56512)
0809 00:37:32.585302 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.585318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.585381 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.585661 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.585662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.585663 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:32.585719 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.585722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.586478 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.586479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.587093 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.587095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.587521 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.587522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.587604 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.587619 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:32.587912 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:32.587921 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.587963 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.588030 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 00:37:32.590149 9989 raft_consensus.cc:1261] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.590234 9676 raft_consensus.cc:1261] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.590518 10070 consensus_queue.cc:1035] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:32.590673 10078 consensus_queue.cc:1035] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:32.592020 10072 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: TxnStatusTablet state changed. Reason: New leader 3ed80988d52c429192ff22c07b6e440b. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.592154 10072 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.592109 10076 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: TxnStatusTablet state changed. Reason: New leader 3ed80988d52c429192ff22c07b6e440b. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.592200 10076 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.593583 10070 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.593678 10070 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:32.593675 10076 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.593756 10076 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P dabb54e2bcbd47108148358285282809: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.593782 10070 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.593832 10070 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P 3ed80988d52c429192ff22c07b6e440b: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:32.593897 10095 mvcc.cc:205] Tried to move back new op lower bound from 7316422052206452736 to 7316422052146098176. Current Snapshot: MvccSnapshot[applied={T|T < 7316422052206452736}]
I20260809 00:37:32.594111 10097 mvcc.cc:205] Tried to move back new op lower bound from 7316422052206452736 to 7316422052146098176. Current Snapshot: MvccSnapshot[applied={T|T < 7316422052206452736}]
I20260809 00:37:32.594471 10072 tablet_replica.cc:439] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } } }
I20260809 00:37:32.594571 10072 tablet_replica.cc:442] T 062a4adb716f450a9ee2da1f89b5304a P e1d6b14ca3044a56b8e58979ae323c8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:32.594918 10096 mvcc.cc:205] Tried to move back new op lower bound from 7316422052206452736 to 7316422052146098176. Current Snapshot: MvccSnapshot[applied={T|T < 7316422052206452736 or (T in {7316422052206452736})}]
I20260809 00:37:32.600628 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.597733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.7.105.66:43363)
0809 00:37:32.597780 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.597783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.597919 (+ 136us) server_negotiation.cc:407] Connection header received
0809 00:37:32.597996 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.597996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.598015 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.598069 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:32.598186 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.598649 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.599892 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.600130 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.600191 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.600374 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:32.600401 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:32.600440 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.600500 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.600517 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:37:32.606004 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.603332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:52809 (local address 127.7.105.66:43363)
0809 00:37:32.603401 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:32.603403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:32.603460 (+ 57us) server_negotiation.cc:407] Connection header received
0809 00:37:32.603530 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:32.603531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:32.603540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:32.603585 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:32.603968 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.604489 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.605270 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.605462 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.605540 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.605726 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:32.605756 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:32.605818 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:32.605876 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:37:32.605916 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":51}
I20260809 00:37:32.606024 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:32.603268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.7.105.66:52809)
0809 00:37:32.603409 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:32.603420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:32.603485 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:32.603586 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:32.603587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:32.603588 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:32.603895 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.603897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.604573 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.604574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.605178 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:32.605180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:32.605541 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:32.605542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:32.605606 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:32.605626 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:32.605818 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:32.605820 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:32.605873 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:37:32.605931 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260809 00:37:32.608474 9989 raft_consensus.cc:1261] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.608628 9676 raft_consensus.cc:1261] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.608757 10110 consensus_queue.cc:1035] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:32.608873 10070 consensus_queue.cc:1035] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:32.621430 9506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:32.621557 9506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:32.622299 9506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Sending DeleteTablet for 3 replicas of tablet b54fa0d597d94b1d91e22c7ccb2b7a67
I20260809 00:37:32.622637 9653 tablet_service.cc:1514] Processing DeleteTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:32 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46324
I20260809 00:37:32.622705 9967 tablet_service.cc:1514] Processing DeleteTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:32 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46282
I20260809 00:37:32.622991 10116 tablet_replica.cc:330] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: stopping tablet replica
I20260809 00:37:32.622982 9808 tablet_service.cc:1514] Processing DeleteTablet for tablet b54fa0d597d94b1d91e22c7ccb2b7a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:32 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:56504
I20260809 00:37:32.623121 10116 raft_consensus.cc:2229] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:32.623194 10116 raft_consensus.cc:2258] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:32.623275 10117 tablet_replica.cc:330] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: stopping tablet replica
I20260809 00:37:32.623378 10117 raft_consensus.cc:2229] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:32.623492 10117 raft_consensus.cc:2258] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:32.623466 9506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:32.623622 9506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:32.623867 10117 ts_tablet_manager.cc:1905] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:32.624949 9653 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc0118f14944fa3820d01345465e8af (DEFAULT_TABLE table=test-table [id=43ac942dc91049bfbe10d4b65ade9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.625098 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc0118f14944fa3820d01345465e8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.625672 10117 ts_tablet_manager.cc:1918] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:32.625761 10117 log.cc:1196] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/b54fa0d597d94b1d91e22c7ccb2b7a67
I20260809 00:37:32.626019 10117 ts_tablet_manager.cc:1939] T b54fa0d597d94b1d91e22c7ccb2b7a67 P 3ed80988d52c429192ff22c07b6e440b: Deleting consensus metadata
I20260809 00:37:32.626524 10071 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Bootstrap starting.
I20260809 00:37:32.626755 9967 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc0118f14944fa3820d01345465e8af (DEFAULT_TABLE table=test-table [id=43ac942dc91049bfbe10d4b65ade9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.626941 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc0118f14944fa3820d01345465e8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.627223 10071 tablet_bootstrap.cc:654] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.627876 10071 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: No bootstrap required, opened a new log
I20260809 00:37:32.627936 10071 ts_tablet_manager.cc:1397] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:32.628127 10071 raft_consensus.cc:348] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.628180 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc0118f14944fa3820d01345465e8af (DEFAULT_TABLE table=test-table [id=43ac942dc91049bfbe10d4b65ade9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:32.628211 10071 raft_consensus.cc:374] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.628232 10071 raft_consensus.cc:729] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb54e2bcbd47108148358285282809, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.628286 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc0118f14944fa3820d01345465e8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:32.628281 10071 consensus_queue.cc:260] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.628376 10066 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Bootstrap starting.
I20260809 00:37:32.628382 10071 ts_tablet_manager.cc:1428] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:32.628785 10066 tablet_bootstrap.cc:654] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.629487 10066 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: No bootstrap required, opened a new log
I20260809 00:37:32.629534 10066 ts_tablet_manager.cc:1397] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:32.629688 10067 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Bootstrap starting.
I20260809 00:37:32.629678 10066 raft_consensus.cc:348] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.629745 10066 raft_consensus.cc:374] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.629801 10066 raft_consensus.cc:729] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.629863 10066 consensus_queue.cc:260] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.629983 10066 ts_tablet_manager.cc:1428] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:32.630056 9492 catalog_manager.cc:4841] TS 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363): tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (table test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]) successfully deleted
I20260809 00:37:32.630133 10067 tablet_bootstrap.cc:654] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:32.630434 10116 ts_tablet_manager.cc:1905] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:32.630863 10067 tablet_bootstrap.cc:492] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: No bootstrap required, opened a new log
I20260809 00:37:32.630940 10067 ts_tablet_manager.cc:1397] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:32.631093 10067 raft_consensus.cc:348] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.631183 10067 raft_consensus.cc:374] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:32.631222 10067 raft_consensus.cc:729] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Initialized, Role: FOLLOWER
I20260809 00:37:32.631274 10067 consensus_queue.cc:260] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.631374 10067 ts_tablet_manager.cc:1428] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:32.632045 10116 ts_tablet_manager.cc:1918] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:32.632157 10116 log.cc:1196] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/b54fa0d597d94b1d91e22c7ccb2b7a67
I20260809 00:37:32.632310 10115 tablet_replica.cc:330] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: stopping tablet replica
I20260809 00:37:32.632436 10116 ts_tablet_manager.cc:1939] T b54fa0d597d94b1d91e22c7ccb2b7a67 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting consensus metadata
I20260809 00:37:32.632457 10115 raft_consensus.cc:2229] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:32.632665 10115 raft_consensus.cc:2258] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:32.632895 9492 catalog_manager.cc:4841] TS e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517): tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (table test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]) successfully deleted
I20260809 00:37:32.633170 10115 ts_tablet_manager.cc:1905] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:32.634863 10115 ts_tablet_manager.cc:1918] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:32.634963 10115 log.cc:1196] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/b54fa0d597d94b1d91e22c7ccb2b7a67
I20260809 00:37:32.635263 10115 ts_tablet_manager.cc:1939] T b54fa0d597d94b1d91e22c7ccb2b7a67 P dabb54e2bcbd47108148358285282809: Deleting consensus metadata
I20260809 00:37:32.635746 9492 catalog_manager.cc:4841] TS dabb54e2bcbd47108148358285282809 (127.7.105.65:42153): tablet b54fa0d597d94b1d91e22c7ccb2b7a67 (table test-table [id=cb3e4fc830df44aab01a5e6bc51e73ea]) successfully deleted
I20260809 00:37:32.719296 10078 raft_consensus.cc:482] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:32.719429 10078 raft_consensus.cc:504] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.719651 10078 leader_election.cc:288] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.719918 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc0118f14944fa3820d01345465e8af" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" is_pre_election: true
I20260809 00:37:32.719930 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc0118f14944fa3820d01345465e8af" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809" is_pre_election: true
I20260809 00:37:32.720034 9989 raft_consensus.cc:2454] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.720078 9676 raft_consensus.cc:2454] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 0.
I20260809 00:37:32.720238 9756 leader_election.cc:302] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:32.720360 10078 raft_consensus.cc:2781] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:32.720403 10078 raft_consensus.cc:482] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:32.720430 10078 raft_consensus.cc:3037] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.721171 10078 raft_consensus.cc:504] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.721312 10078 leader_election.cc:288] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Requested vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517)
I20260809 00:37:32.721467 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc0118f14944fa3820d01345465e8af" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809"
I20260809 00:37:32.721494 9989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc0118f14944fa3820d01345465e8af" candidate_uuid: "3ed80988d52c429192ff22c07b6e440b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d6b14ca3044a56b8e58979ae323c8f"
I20260809 00:37:32.721563 9989 raft_consensus.cc:3037] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.721563 9676 raft_consensus.cc:3037] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:32.722318 9989 raft_consensus.cc:2454] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.722319 9676 raft_consensus.cc:2454] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed80988d52c429192ff22c07b6e440b in term 1.
I20260809 00:37:32.722501 9756 leader_election.cc:302] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, dabb54e2bcbd47108148358285282809; no voters:
I20260809 00:37:32.722604 10078 raft_consensus.cc:2781] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:32.722661 10078 raft_consensus.cc:686] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 1 LEADER]: Becoming Leader. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Running, Role: LEADER
I20260809 00:37:32.722719 10078 consensus_queue.cc:237] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:32.723416 9505 catalog_manager.cc:5495] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed80988d52c429192ff22c07b6e440b (127.7.105.66). New cstate: current_term: 1 leader_uuid: "3ed80988d52c429192ff22c07b6e440b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:32.752327 9676 raft_consensus.cc:1261] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.752336 9989 raft_consensus.cc:1261] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Refusing update from remote peer 3ed80988d52c429192ff22c07b6e440b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:32.752571 10110 consensus_queue.cc:1035] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:32.752710 10070 consensus_queue.cc:1035] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:37:32.758070 10095 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:37:34.278393 9506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:34.278524 9506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:34.279422 9506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Sending DeleteTablet for 3 replicas of tablet 0fc0118f14944fa3820d01345465e8af
I20260809 00:37:34.279735 9653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fc0118f14944fa3820d01345465e8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46324
I20260809 00:37:34.279984 9808 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fc0118f14944fa3820d01345465e8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:56504
I20260809 00:37:34.280002 10147 tablet_replica.cc:330] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: stopping tablet replica
I20260809 00:37:34.280107 10147 raft_consensus.cc:2229] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:34.280177 10147 raft_consensus.cc:2258] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.280195 10148 tablet_replica.cc:330] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: stopping tablet replica
I20260809 00:37:34.280247 10148 raft_consensus.cc:2229] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:34.280303 10148 raft_consensus.cc:2258] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.280452 10147 ts_tablet_manager.cc:1905] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.280586 10148 ts_tablet_manager.cc:1905] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.281967 10147 ts_tablet_manager.cc:1918] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:37:34.282040 10147 log.cc:1196] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/0fc0118f14944fa3820d01345465e8af
I20260809 00:37:34.282305 10147 ts_tablet_manager.cc:1939] T 0fc0118f14944fa3820d01345465e8af P dabb54e2bcbd47108148358285282809: Deleting consensus metadata
I20260809 00:37:34.282423 9967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fc0118f14944fa3820d01345465e8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46282
I20260809 00:37:34.282619 10148 ts_tablet_manager.cc:1918] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:37:34.282717 10148 log.cc:1196] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/0fc0118f14944fa3820d01345465e8af
I20260809 00:37:34.282773 9492 catalog_manager.cc:4841] TS dabb54e2bcbd47108148358285282809 (127.7.105.65:42153): tablet 0fc0118f14944fa3820d01345465e8af (table test-table [id=43ac942dc91049bfbe10d4b65ade9759]) successfully deleted
I20260809 00:37:34.283036 10148 ts_tablet_manager.cc:1939] T 0fc0118f14944fa3820d01345465e8af P 3ed80988d52c429192ff22c07b6e440b: Deleting consensus metadata
I20260809 00:37:34.283396 9492 catalog_manager.cc:4841] TS 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363): tablet 0fc0118f14944fa3820d01345465e8af (table test-table [id=43ac942dc91049bfbe10d4b65ade9759]) successfully deleted
I20260809 00:37:34.283921 10151 tablet_replica.cc:330] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: stopping tablet replica
I20260809 00:37:34.284006 10151 raft_consensus.cc:2229] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:34.284082 10151 raft_consensus.cc:2258] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.284332 10151 ts_tablet_manager.cc:1905] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.286304 10151 ts_tablet_manager.cc:1918] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:37:34.286373 10151 log.cc:1196] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/0fc0118f14944fa3820d01345465e8af
I20260809 00:37:34.286628 10151 ts_tablet_manager.cc:1939] T 0fc0118f14944fa3820d01345465e8af P e1d6b14ca3044a56b8e58979ae323c8f: Deleting consensus metadata
I20260809 00:37:34.287253 9492 catalog_manager.cc:4841] TS e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517): tablet 0fc0118f14944fa3820d01345465e8af (table test-table [id=43ac942dc91049bfbe10d4b65ade9759]) successfully deleted
Aug 09 00:37:34 dist-test-slave-1zqn krb5kdc[9457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235854, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:37:34 dist-test-slave-1zqn krb5kdc[9457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235854, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:34.311321 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.299944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.7.105.126:44887)
0809 00:37:34.300171 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.300173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.300183 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:34.300209 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.300210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.300222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.300294 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:34.300837 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.301442 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.302188 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.302403 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.302463 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.308742 (+ 6279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:34.308743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:34.308744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:34.308751 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:34.310236 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:34.310623 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:34.310624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:34.310624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:34.310679 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:34.310836 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:34.310836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:34.310837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:34.310940 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:34.311015 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.311132 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.311202 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":128,"threads_started":1}
I20260809 00:37:34.312997 9506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:34.313179 9506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:34.314637 9653 tablet_service.cc:1467] Processing CreateTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c (DEFAULT_TABLE table=test-table [id=57552602880640c3aa0944604c170dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.314637 9967 tablet_service.cc:1467] Processing CreateTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c (DEFAULT_TABLE table=test-table [id=57552602880640c3aa0944604c170dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.314791 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a03f03a33e418cac0d0c5ae7e3417c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.314795 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a03f03a33e418cac0d0c5ae7e3417c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.316190 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c (DEFAULT_TABLE table=test-table [id=57552602880640c3aa0944604c170dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.316247 10159 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Bootstrap starting.
I20260809 00:37:34.316334 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a03f03a33e418cac0d0c5ae7e3417c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.316346 10160 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Bootstrap starting.
I20260809 00:37:34.316619 10159 tablet_bootstrap.cc:654] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.316665 10160 tablet_bootstrap.cc:654] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.317242 10159 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: No bootstrap required, opened a new log
I20260809 00:37:34.317247 10160 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: No bootstrap required, opened a new log
I20260809 00:37:34.317279 10159 ts_tablet_manager.cc:1397] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:34.317299 10160 ts_tablet_manager.cc:1397] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:34.317404 10159 raft_consensus.cc:348] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.317454 10159 raft_consensus.cc:374] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.317469 10159 raft_consensus.cc:729] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.317448 10160 raft_consensus.cc:348] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.317512 10160 raft_consensus.cc:374] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.317528 10159 consensus_queue.cc:260] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.317570 10160 raft_consensus.cc:729] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb54e2bcbd47108148358285282809, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.317621 10160 consensus_queue.cc:260] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.317731 10159 ts_tablet_manager.cc:1428] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:34.317814 10160 ts_tablet_manager.cc:1428] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:34.317945 10167 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Bootstrap starting.
I20260809 00:37:34.318344 10167 tablet_bootstrap.cc:654] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.319002 10167 tablet_bootstrap.cc:492] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: No bootstrap required, opened a new log
I20260809 00:37:34.319047 10167 ts_tablet_manager.cc:1397] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:34.319175 10167 raft_consensus.cc:348] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.319242 10167 raft_consensus.cc:374] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.319257 10167 raft_consensus.cc:729] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.319324 10167 consensus_queue.cc:260] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.319437 10167 ts_tablet_manager.cc:1428] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:34.443184 10164 raft_consensus.cc:482] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:34.443296 10164 raft_consensus.cc:504] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.443598 10164 leader_election.cc:288] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363)
I20260809 00:37:34.447700 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.443810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42153 (local address 127.7.105.67:51139)
0809 00:37:34.444002 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:34.444015 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:34.444105 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:34.444307 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:34.444309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:34.444311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:34.444399 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.444401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.445403 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.445404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.446859 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.446862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.447492 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.447493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.447547 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.447556 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 00:37:34.447599 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 00:37:34.447631 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":49,"thread_start_us":130,"threads_started":1}
I20260809 00:37:34.447745 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.443904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:51139 (local address 127.7.105.65:42153)
0809 00:37:34.444123 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.444125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.444139 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:37:34.444178 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.444179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.444194 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.444291 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:34.444488 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.445319 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.446951 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.447412 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.447487 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.447497 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.447626 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.447668 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260809 00:37:34.448041 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a03f03a33e418cac0d0c5ae7e3417c" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809" is_pre_election: true
I20260809 00:37:34.448221 9676 raft_consensus.cc:2454] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 0.
I20260809 00:37:34.448310 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.444185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.7.105.67:46351)
0809 00:37:34.444335 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:34.444344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:34.444390 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:34.444578 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:34.444579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:34.444581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:34.444632 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.444633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.445481 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.445482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.447174 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.447177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.447947 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.447948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.448120 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.448126 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:37:34.448178 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:37:34.448221 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":22,"thread_start_us":95,"threads_started":1}
I20260809 00:37:34.448311 10176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.444208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:46351 (local address 127.7.105.66:43363)
0809 00:37:34.444378 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.444380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.444387 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:34.444475 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.444475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.444487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.444564 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:34.444700 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.445408 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.447251 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.447810 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.447885 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.447896 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.448196 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.448239 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260809 00:37:34.448441 9915 leader_election.cc:302] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dabb54e2bcbd47108148358285282809, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:34.448592 10164 raft_consensus.cc:2781] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:34.448645 10164 raft_consensus.cc:482] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:34.448668 10164 raft_consensus.cc:3037] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.448771 9832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a03f03a33e418cac0d0c5ae7e3417c" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed80988d52c429192ff22c07b6e440b" is_pre_election: true
I20260809 00:37:34.448915 9832 raft_consensus.cc:2454] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 0.
I20260809 00:37:34.449538 10164 raft_consensus.cc:504] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.449680 10164 leader_election.cc:288] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 election: Requested vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363)
I20260809 00:37:34.449869 9832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a03f03a33e418cac0d0c5ae7e3417c" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed80988d52c429192ff22c07b6e440b"
I20260809 00:37:34.449857 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a03f03a33e418cac0d0c5ae7e3417c" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809"
I20260809 00:37:34.449932 9832 raft_consensus.cc:3037] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.449950 9676 raft_consensus.cc:3037] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.450570 9676 raft_consensus.cc:2454] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 1.
I20260809 00:37:34.450627 9832 raft_consensus.cc:2454] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 1.
I20260809 00:37:34.450778 9915 leader_election.cc:302] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dabb54e2bcbd47108148358285282809, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:34.450886 10164 raft_consensus.cc:2781] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:34.451064 10164 raft_consensus.cc:686] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 1 LEADER]: Becoming Leader. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Running, Role: LEADER
I20260809 00:37:34.451149 10164 consensus_queue.cc:237] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.451902 9506 catalog_manager.cc:5495] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f reported cstate change: term changed from 0 to 1, leader changed from <none> to e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67). New cstate: current_term: 1 leader_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } health_report { overall_health: HEALTHY } } }
I20260809 00:37:34.465687 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.463301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.7.105.67:41517)
0809 00:37:34.463470 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.463472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.463478 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:34.463511 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.463511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.463531 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.463604 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:34.463707 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.464280 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.464981 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.465210 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.465264 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.465387 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:34.465498 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:34.465550 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.465603 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.465619 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260809 00:37:34.470412 10176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.468116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:40679 (local address 127.7.105.66:43363)
0809 00:37:34.468160 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.468162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.468219 (+ 57us) server_negotiation.cc:407] Connection header received
0809 00:37:34.468268 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.468269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.468279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.468322 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:34.468463 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.468917 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.469726 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.469980 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.470041 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.470239 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:34.470265 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:34.470299 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.470342 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.470359 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:37:34.470428 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.467978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.7.105.67:40679)
0809 00:37:34.468165 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:34.468180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:34.468231 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:34.468323 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:34.468324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:34.468325 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:34.468380 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.468381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.468993 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.468994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.469657 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.469660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.470049 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.470083 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.470154 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.470168 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:34.470307 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:34.470308 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:34.470334 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 00:37:34.470364 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260809 00:37:34.473003 9676 raft_consensus.cc:1261] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Refusing update from remote peer e1d6b14ca3044a56b8e58979ae323c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:34.473120 9832 raft_consensus.cc:1261] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Refusing update from remote peer e1d6b14ca3044a56b8e58979ae323c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:34.473316 10177 consensus_queue.cc:1035] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:34.473433 10164 consensus_queue.cc:1035] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:34.480784 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.478373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:46947 (local address 127.7.105.67:41517)
0809 00:37:34.478476 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:34.478477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:34.478553 (+ 76us) server_negotiation.cc:407] Connection header received
0809 00:37:34.478604 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:34.478605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:34.478616 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:34.478678 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:34.478782 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.479234 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.479976 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.480286 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.480366 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.480545 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:34.480579 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:34.480633 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:34.480694 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:37:34.480713 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 00:37:34.480810 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:34.478340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:41517 (local address 127.7.105.66:46947)
0809 00:37:34.478479 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:34.478491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:34.478548 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:34.478684 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:34.478685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:34.478687 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:34.478732 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.478734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.479311 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.479312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.479879 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:34.479881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:34.480386 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:34.480387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:34.480459 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:34.480464 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:34.480636 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:34.480639 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:34.480681 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 00:37:34.480731 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
I20260809 00:37:34.487803 9506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:34.487941 9506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:34.488670 9506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Sending DeleteTablet for 3 replicas of tablet 34a03f03a33e418cac0d0c5ae7e3417c
I20260809 00:37:34.488920 9653 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46324
I20260809 00:37:34.489024 9808 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:56504
I20260809 00:37:34.489045 10147 tablet_replica.cc:330] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: stopping tablet replica
I20260809 00:37:34.489108 10147 raft_consensus.cc:2229] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:34.489130 10148 tablet_replica.cc:330] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: stopping tablet replica
I20260809 00:37:34.489149 10147 raft_consensus.cc:2258] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.489132 9967 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a03f03a33e418cac0d0c5ae7e3417c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46282
I20260809 00:37:34.489180 10148 raft_consensus.cc:2229] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:34.489235 10148 raft_consensus.cc:2258] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.489336 10151 tablet_replica.cc:330] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: stopping tablet replica
I20260809 00:37:34.489395 10151 raft_consensus.cc:2229] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:34.489490 10151 raft_consensus.cc:2258] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:34.489487 10147 ts_tablet_manager.cc:1905] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.489526 9506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:34.489682 9506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:34.489921 10151 ts_tablet_manager.cc:1905] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.490010 10148 ts_tablet_manager.cc:1905] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:34.491511 10151 ts_tablet_manager.cc:1918] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:34.491511 10148 ts_tablet_manager.cc:1918] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:34.491567 10148 log.cc:1196] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/34a03f03a33e418cac0d0c5ae7e3417c
I20260809 00:37:34.491575 10151 log.cc:1196] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/34a03f03a33e418cac0d0c5ae7e3417c
I20260809 00:37:34.491792 10148 ts_tablet_manager.cc:1939] T 34a03f03a33e418cac0d0c5ae7e3417c P 3ed80988d52c429192ff22c07b6e440b: Deleting consensus metadata
I20260809 00:37:34.491792 10151 ts_tablet_manager.cc:1939] T 34a03f03a33e418cac0d0c5ae7e3417c P e1d6b14ca3044a56b8e58979ae323c8f: Deleting consensus metadata
I20260809 00:37:34.492632 9492 catalog_manager.cc:4841] TS 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363): tablet 34a03f03a33e418cac0d0c5ae7e3417c (table test-table [id=57552602880640c3aa0944604c170dce]) successfully deleted
I20260809 00:37:34.492717 10147 ts_tablet_manager.cc:1918] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:34.492789 10147 log.cc:1196] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/34a03f03a33e418cac0d0c5ae7e3417c
I20260809 00:37:34.493031 10147 ts_tablet_manager.cc:1939] T 34a03f03a33e418cac0d0c5ae7e3417c P dabb54e2bcbd47108148358285282809: Deleting consensus metadata
I20260809 00:37:34.493054 9653 tablet_service.cc:1467] Processing CreateTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 (DEFAULT_TABLE table=test-table [id=ad44848db98d4ee09bfe7e5913318e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.493110 9967 tablet_service.cc:1467] Processing CreateTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 (DEFAULT_TABLE table=test-table [id=ad44848db98d4ee09bfe7e5913318e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.493167 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b9c3256dc54288a9b6925be8fb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.493222 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b9c3256dc54288a9b6925be8fb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.494227 10159 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Bootstrap starting.
I20260809 00:37:34.494318 10160 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809: Bootstrap starting.
I20260809 00:37:34.494319 9492 catalog_manager.cc:4841] TS e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517): tablet 34a03f03a33e418cac0d0c5ae7e3417c (table test-table [id=57552602880640c3aa0944604c170dce]) successfully deleted
I20260809 00:37:34.494408 9492 catalog_manager.cc:4841] TS dabb54e2bcbd47108148358285282809 (127.7.105.65:42153): tablet 34a03f03a33e418cac0d0c5ae7e3417c (table test-table [id=57552602880640c3aa0944604c170dce]) successfully deleted
I20260809 00:37:34.494608 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 (DEFAULT_TABLE table=test-table [id=ad44848db98d4ee09bfe7e5913318e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:34.494647 10159 tablet_bootstrap.cc:654] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.494707 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b9c3256dc54288a9b6925be8fb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:34.494874 10160 tablet_bootstrap.cc:654] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.495504 10160 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809: No bootstrap required, opened a new log
I20260809 00:37:34.495572 10160 ts_tablet_manager.cc:1397] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:34.495733 10160 raft_consensus.cc:348] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.495795 10160 raft_consensus.cc:374] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.495814 10160 raft_consensus.cc:729] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb54e2bcbd47108148358285282809, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.495859 10160 consensus_queue.cc:260] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.495952 10160 ts_tablet_manager.cc:1428] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:34.496120 10167 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: Bootstrap starting.
I20260809 00:37:34.496465 10167 tablet_bootstrap.cc:654] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:34.496865 10159 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: No bootstrap required, opened a new log
I20260809 00:37:34.496918 10159 ts_tablet_manager.cc:1397] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:37:34.497089 10159 raft_consensus.cc:348] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.497161 10159 raft_consensus.cc:374] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.497185 10159 raft_consensus.cc:729] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.497236 10159 consensus_queue.cc:260] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.497368 10159 ts_tablet_manager.cc:1428] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:34.498003 10167 tablet_bootstrap.cc:492] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: No bootstrap required, opened a new log
I20260809 00:37:34.498063 10167 ts_tablet_manager.cc:1397] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:34.498170 10167 raft_consensus.cc:348] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.498240 10167 raft_consensus.cc:374] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:34.498261 10167 raft_consensus.cc:729] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed80988d52c429192ff22c07b6e440b, State: Initialized, Role: FOLLOWER
I20260809 00:37:34.498317 10167 consensus_queue.cc:260] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.498409 10167 ts_tablet_manager.cc:1428] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:34.732792 10177 raft_consensus.cc:482] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:34.732935 10177 raft_consensus.cc:504] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.733089 10177 leader_election.cc:288] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363)
I20260809 00:37:34.733364 9832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b9c3256dc54288a9b6925be8fb9941" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed80988d52c429192ff22c07b6e440b" is_pre_election: true
I20260809 00:37:34.733471 9832 raft_consensus.cc:2454] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 0.
I20260809 00:37:34.733448 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b9c3256dc54288a9b6925be8fb9941" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809" is_pre_election: true
I20260809 00:37:34.733577 9676 raft_consensus.cc:2454] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 0.
I20260809 00:37:34.733652 9915 leader_election.cc:302] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed80988d52c429192ff22c07b6e440b, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:34.733764 10177 raft_consensus.cc:2781] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:34.733800 10177 raft_consensus.cc:482] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:34.733814 10177 raft_consensus.cc:3037] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.734586 10177 raft_consensus.cc:504] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.734719 10177 leader_election.cc:288] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 election: Requested vote from peers dabb54e2bcbd47108148358285282809 (127.7.105.65:42153), 3ed80988d52c429192ff22c07b6e440b (127.7.105.66:43363)
I20260809 00:37:34.734910 9832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b9c3256dc54288a9b6925be8fb9941" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed80988d52c429192ff22c07b6e440b"
I20260809 00:37:34.734916 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b9c3256dc54288a9b6925be8fb9941" candidate_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabb54e2bcbd47108148358285282809"
I20260809 00:37:34.734999 9676 raft_consensus.cc:3037] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.735000 9832 raft_consensus.cc:3037] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:34.735653 9832 raft_consensus.cc:2454] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 1.
I20260809 00:37:34.735697 9676 raft_consensus.cc:2454] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1d6b14ca3044a56b8e58979ae323c8f in term 1.
I20260809 00:37:34.735849 9915 leader_election.cc:302] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dabb54e2bcbd47108148358285282809, e1d6b14ca3044a56b8e58979ae323c8f; no voters:
I20260809 00:37:34.735978 10177 raft_consensus.cc:2781] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:34.736043 10177 raft_consensus.cc:686] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 LEADER]: Becoming Leader. State: Replica: e1d6b14ca3044a56b8e58979ae323c8f, State: Running, Role: LEADER
I20260809 00:37:34.736132 10177 consensus_queue.cc:237] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } }
I20260809 00:37:34.736814 9506 catalog_manager.cc:5495] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f reported cstate change: term changed from 0 to 1, leader changed from <none> to e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67). New cstate: current_term: 1 leader_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1d6b14ca3044a56b8e58979ae323c8f" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 41517 } health_report { overall_health: HEALTHY } } }
I20260809 00:37:34.784147 9832 raft_consensus.cc:1261] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Refusing update from remote peer e1d6b14ca3044a56b8e58979ae323c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:34.784154 9676 raft_consensus.cc:1261] T 33b9c3256dc54288a9b6925be8fb9941 P dabb54e2bcbd47108148358285282809 [term 1 FOLLOWER]: Refusing update from remote peer e1d6b14ca3044a56b8e58979ae323c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:34.784415 10164 consensus_queue.cc:1035] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed80988d52c429192ff22c07b6e440b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:34.784550 10177 consensus_queue.cc:1035] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabb54e2bcbd47108148358285282809" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:37:34.789378 10129 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:37:36.306842 9506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:36.306947 9506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:36.307776 9506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6546377d0f6a442c80c71772002df19d: Sending DeleteTablet for 3 replicas of tablet 33b9c3256dc54288a9b6925be8fb9941
I20260809 00:37:36.308122 9653 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:36 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46324
I20260809 00:37:36.308113 9808 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:36 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:56504
I20260809 00:37:36.308130 9967 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b9c3256dc54288a9b6925be8fb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:36 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46282
I20260809 00:37:36.308276 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9584
I20260809 00:37:36.308323 10228 tablet_replica.cc:330] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: stopping tablet replica
I20260809 00:37:36.311110 10228 raft_consensus.cc:2229] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:36.311206 10228 raft_consensus.cc:2258] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:36.311447 10228 ts_tablet_manager.cc:1905] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:36.313668 10228 ts_tablet_manager.cc:1918] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:37:36.313750 10228 log.cc:1196] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/33b9c3256dc54288a9b6925be8fb9941
I20260809 00:37:36.314007 10228 ts_tablet_manager.cc:1939] T 33b9c3256dc54288a9b6925be8fb9941 P e1d6b14ca3044a56b8e58979ae323c8f: Deleting consensus metadata
I20260809 00:37:36.314352 9492 catalog_manager.cc:4841] TS e1d6b14ca3044a56b8e58979ae323c8f (127.7.105.67:41517): tablet 33b9c3256dc54288a9b6925be8fb9941 (table test-table [id=ad44848db98d4ee09bfe7e5913318e86]) successfully deleted
I20260809 00:37:36.316176 10227 tablet_replica.cc:330] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: stopping tablet replica
I20260809 00:37:36.316262 10227 raft_consensus.cc:2229] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:36.316313 10227 raft_consensus.cc:2258] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:36.316563 10227 ts_tablet_manager.cc:1905] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:37:36.317737 9492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:42153 (error 108)
I20260809 00:37:36.317953 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9736
I20260809 00:37:36.318347 10227 ts_tablet_manager.cc:1918] T 33b9c3256dc54288a9b6925be8fb9941 P 3ed80988d52c429192ff22c07b6e440b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:37:36.324304 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:36.318155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42153 (local address 127.0.0.1:46330)
0809 00:37:36.324200 (+ 6045us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:36.324263 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6014,"thread_start_us":64,"threads_started":1}
W20260809 00:37:36.324476 9492 catalog_manager.cc:4584] TS dabb54e2bcbd47108148358285282809 (127.7.105.65:42153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33b9c3256dc54288a9b6925be8fb9941: Network error: Client connection negotiation failed: client connection to 127.7.105.65:42153: connect: Connection refused (error 111)
I20260809 00:37:36.326623 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9896
I20260809 00:37:36.326634 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:36.326518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43363 (local address 127.0.0.1:56532)
0809 00:37:36.326575 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:36.326601 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.66:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:37:36.337350 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 9472
2026-08-09T00:37:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6359 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:36 dist-test-slave-1zqn krb5kdc[10234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:36 dist-test-slave-1zqn krb5kdc[10234](info): set up 2 sockets
Aug 09 00:37:36 dist-test-slave-1zqn krb5kdc[10234](info): commencing operation
krb5kdc: starting...
W20260809 00:37:37.438061 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:37 dist-test-slave-1zqn krb5kdc[10234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:37.571396 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36451
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:37.654835 10248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:37.655066 10248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:37.655090 10248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:37.656343 10248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:37.656414 10248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:37.656430 10248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:37.656443 10248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:37.656456 10248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:37.658033 10248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:37.658304 10248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:37.658548 10248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:37.663808 10257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:37.663832 10248 server_base.cc:1034] running on GCE node
W20260809 00:37:37.663753 10254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:37.663792 10255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:37.664318 10248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:37.664621 10248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:37.665782 10248 hybrid_clock.cc:648] HybridClock initialized: now 1786235857665772 us; error 37 us; skew 500 ppm
Aug 09 00:37:37 dist-test-slave-1zqn krb5kdc[10234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:37.668447 10248 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:37.668936 10248 webserver.cc:460] Webserver started at http://127.7.105.126:45207/ using document root <none> and password file <none>
I20260809 00:37:37.669189 10248 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:37.669255 10248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:37.669395 10248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:37.670469 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "2ea3e29e3f834b079d1d57fdb265f5b4"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "1233088c1cfdcc8a63c780763dd7d545"
server_key_iv: "ea469e62192952bd4c1ac5b482755e75"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.670876 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "2ea3e29e3f834b079d1d57fdb265f5b4"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "1233088c1cfdcc8a63c780763dd7d545"
server_key_iv: "ea469e62192952bd4c1ac5b482755e75"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.672312 10248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:37.673067 10265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:37.673244 10248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:37.673334 10248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "2ea3e29e3f834b079d1d57fdb265f5b4"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "1233088c1cfdcc8a63c780763dd7d545"
server_key_iv: "ea469e62192952bd4c1ac5b482755e75"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.673413 10248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:37.684883 10248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:37.687218 10248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:37.687384 10248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:37.691766 10248 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:36451
I20260809 00:37:37.691803 10325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:36451 every 8 connection(s)
I20260809 00:37:37.692235 10248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:37.693178 10326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:37.695638 10326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4: Bootstrap starting.
I20260809 00:37:37.696241 10326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:37.696501 10326 log.cc:824] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:37.696969 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10248
I20260809 00:37:37.697081 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:37.697268 7589 external_mini_cluster.cc:1428] Setting key 381922a636d7e6a049edaa5c17fdff6f
I20260809 00:37:37.697319 10326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4: No bootstrap required, opened a new log
I20260809 00:37:37.699254 10326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } }
I20260809 00:37:37.699424 10326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:37.699465 10326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea3e29e3f834b079d1d57fdb265f5b4, State: Initialized, Role: FOLLOWER
I20260809 00:37:37.699661 10326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } }
I20260809 00:37:37.699759 10326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:37.699800 10326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:37.699877 10326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:37.700603 10326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } }
I20260809 00:37:37.700696 10326 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea3e29e3f834b079d1d57fdb265f5b4; no voters:
I20260809 00:37:37.700881 10326 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:37.700932 10331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:37.701143 10326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:37.701140 10331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea3e29e3f834b079d1d57fdb265f5b4, State: Running, Role: LEADER
I20260809 00:37:37.701387 10331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } }
I20260809 00:37:37.701781 10332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } } }
I20260809 00:37:37.701849 10332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:37.701807 10334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea3e29e3f834b079d1d57fdb265f5b4. Latest consensus state: current_term: 1 leader_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea3e29e3f834b079d1d57fdb265f5b4" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36451 } } }
I20260809 00:37:37.701910 10334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:37.702759 10341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:37.703320 10341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:37.705647 10341 catalog_manager.cc:1347] Generated new cluster ID: 9150c9249f98493cb6632795d3d47f4c
I20260809 00:37:37.705696 10341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:37 dist-test-slave-1zqn krb5kdc[10234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:37.710285 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:37.697813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.7.105.126:36451)
0809 00:37:37.698006 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:37.698014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:37.698041 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:37:37.699236 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:37.699244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:37.699296 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:37.699436 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:37.700284 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.702038 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.703157 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.703641 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.703731 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:37.706622 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:37.706626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:37.706626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:37.706638 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:37.708608 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:37.709013 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:37.709015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:37.709016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:37.709144 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:37.709354 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:37.709354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:37.709355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:37.709636 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:37.709780 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:37.709948 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:37:37.710094 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260809 00:37:37.734515 10341 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:37.735122 10341 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:37.738130 10341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea3e29e3f834b079d1d57fdb265f5b4: Generated new TSK 0
I20260809 00:37:37.738327 10341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:37.767338 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:37.852285 10353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:37.852506 10353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:37.852530 10353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:37.853693 10353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:37.853735 10353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:37.853762 10353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:37.855225 10353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:37.855489 10353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:37.855734 10353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:37.860822 10359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:37.860847 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:37.860901 10361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:37.861133 10353 server_base.cc:1034] running on GCE node
I20260809 00:37:37.861397 10353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:37.861683 10353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:37.862848 10353 hybrid_clock.cc:648] HybridClock initialized: now 1786235857862836 us; error 29 us; skew 500 ppm
Aug 09 00:37:37 dist-test-slave-1zqn krb5kdc[10234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:37.865412 10353 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:37.865891 10353 webserver.cc:460] Webserver started at http://127.7.105.65:42329/ using document root <none> and password file <none>
I20260809 00:37:37.866137 10353 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:37.866185 10353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:37.866346 10353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:37.867399 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "3326a44d9e5e49b5ace34a076737cd7f"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "d4029b7d5cd3805a2085c92fb9f2fbe8"
server_key_iv: "706d9039b3d9797866e5324da0bf79cb"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.867771 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "3326a44d9e5e49b5ace34a076737cd7f"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "d4029b7d5cd3805a2085c92fb9f2fbe8"
server_key_iv: "706d9039b3d9797866e5324da0bf79cb"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.869201 10353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:37.869935 10368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:37.870107 10353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:37.870198 10353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "3326a44d9e5e49b5ace34a076737cd7f"
format_stamp: "Formatted at 2026-08-09 00:37:37 on dist-test-slave-1zqn"
server_key: "d4029b7d5cd3805a2085c92fb9f2fbe8"
server_key_iv: "706d9039b3d9797866e5324da0bf79cb"
server_key_version: "encryptionkey@0"
I20260809 00:37:37.870271 10353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:37.882194 10353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:37.884184 10353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:37.884356 10353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:37.884631 10353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:37.885031 10353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:37.885097 10353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:37.885128 10353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:37.885146 10353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:37.891304 10353 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:33395
I20260809 00:37:37.891358 10497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:33395 every 8 connection(s)
I20260809 00:37:37.891687 10353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:37.893165 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10353
I20260809 00:37:37.893267 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:37.893443 7589 external_mini_cluster.cc:1428] Setting key fe28b15776f9aa700aafe30593d8d1c2
Aug 09 00:37:37 dist-test-slave-1zqn krb5kdc[10234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:37:37.902747 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:37.892632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:33965 (local address 127.7.105.126:36451)
0809 00:37:37.892700 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:37.892703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:37.892959 (+ 256us) server_negotiation.cc:407] Connection header received
0809 00:37:37.893705 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:37.893707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:37.893725 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:37.893777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:37.895041 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.895544 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.896788 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.897199 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.897319 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:37.899719 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:37.899722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:37.899722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:37.899730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:37.901276 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:37.901755 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:37.901756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:37.901756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:37.901820 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:37.902054 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:37.902056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:37.902056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:37.902205 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:37.902272 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:37.902555 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 00:37:37.902652 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:37:37.902973 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:37.892633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36451 (local address 127.7.105.65:33965)
0809 00:37:37.892817 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:37.892857 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:37.893591 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:37.893851 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:37.893855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:37.894638 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:37.894932 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:37.894939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.895640 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.895642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:37.896691 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:37.896694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:37.897334 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:37.897335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:37.897481 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:37.898035 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:37.898045 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:37.899614 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:37.901355 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:37.901364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:37.901367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:37.901623 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:37.901882 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:37.901882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:37.901882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:37.901986 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:37.902263 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:37.902272 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:37.902415 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:37:37.902519 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 00:37:37.902699 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260809 00:37:37.903613 10499 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36451
I20260809 00:37:37.903730 10499 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:37.903939 10499 heartbeater.cc:511] Master 127.7.105.126:36451 requested a full tablet report, sending...
I20260809 00:37:37.904565 10286 ts_manager.cc:194] Registered new tserver with Master: 3326a44d9e5e49b5ace34a076737cd7f (127.7.105.65:33395)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
I20260809 00:37:37.905195 10286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:33965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:37.924130 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:38.008994 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:38.009239 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:38.009277 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:38.010470 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:38.010538 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:38.010582 10504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:38.012102 10504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:38.012369 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:38.012612 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:38.017867 10512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:38.017937 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:38.018165 10504 server_base.cc:1034] running on GCE node
W20260809 00:37:38.017867 10509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:38.018489 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:38.018718 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:38.019860 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786235858019841 us; error 36 us; skew 500 ppm
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:38.022504 10504 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:38.022950 10504 webserver.cc:460] Webserver started at http://127.7.105.66:43183/ using document root <none> and password file <none>
I20260809 00:37:38.023180 10504 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:38.023247 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:38.023398 10504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:38.024559 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "f301339545e64cd2917bc19f52ccbd86"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "99b9ab552b69b9e241247d60519beb49"
server_key_iv: "493758e38b1dc13090975da87a2c7c7c"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.024923 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "f301339545e64cd2917bc19f52ccbd86"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "99b9ab552b69b9e241247d60519beb49"
server_key_iv: "493758e38b1dc13090975da87a2c7c7c"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.026295 10504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:37:38.027036 10520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.027207 10504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:38.027311 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "f301339545e64cd2917bc19f52ccbd86"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "99b9ab552b69b9e241247d60519beb49"
server_key_iv: "493758e38b1dc13090975da87a2c7c7c"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.027384 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:38.043533 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:38.046057 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:38.046239 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:38.046506 10504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:38.046837 10504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:38.046897 10504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.046972 10504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:38.047003 10504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.053006 10504 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:34911
I20260809 00:37:38.053103 10645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:34911 every 8 connection(s)
I20260809 00:37:38.053418 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:38.059942 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10504
I20260809 00:37:38.060055 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:38.060303 7589 external_mini_cluster.cc:1428] Setting key b393817f014393c86b0e574a7bb1c163
I20260809 00:37:38.063959 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:38.054123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36451 (local address 127.7.105.66:46033)
0809 00:37:38.054278 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:38.054322 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:38.054801 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:38.055036 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:38.055039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:38.055614 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:38.055811 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:38.055816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.056587 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.056588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:38.057454 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:38.057457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.057810 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.057811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:38.057906 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:38.058557 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:38.058566 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:38.060037 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:38.062397 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:38.062405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:38.062407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:38.062750 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:38.062995 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:38.062996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:38.062996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:38.063136 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:38.063439 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:38.063443 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:38.063569 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 00:37:38.063656 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:37:38.063783 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 00:37:38.063957 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:38.054174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:46033 (local address 127.7.105.126:36451)
0809 00:37:38.054248 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:38.054250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:38.054405 (+ 155us) server_negotiation.cc:407] Connection header received
0809 00:37:38.054897 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:38.054898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:38.054910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:38.054978 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:38.055898 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.056498 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.057542 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.057733 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.057790 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:38.060242 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:38.060244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:38.060244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:38.060251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:38.062300 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:38.062842 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:38.062844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:38.062844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:38.062918 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:38.063209 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:38.063210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:38.063210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:38.063369 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:38.063443 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:38.063711 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 00:37:38.063813 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:37:38.064613 10646 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36451
I20260809 00:37:38.064735 10646 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:38.064921 10646 heartbeater.cc:511] Master 127.7.105.126:36451 requested a full tablet report, sending...
I20260809 00:37:38.065358 10286 ts_manager.cc:194] Registered new tserver with Master: f301339545e64cd2917bc19f52ccbd86 (127.7.105.66:34911)
I20260809 00:37:38.065779 10286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:46033
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:38.092769 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:38.177357 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:38.177698 10651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:38.177755 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:38.178985 10651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:38.179031 10651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:38.179059 10651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:38.180586 10651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:38.180855 10651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:38.181102 10651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:38.186434 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:38.186434 10656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:38.186494 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:38.186766 10651 server_base.cc:1034] running on GCE node
I20260809 00:37:38.186986 10651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:38.187206 10651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:38.188329 10651 hybrid_clock.cc:648] HybridClock initialized: now 1786235858188309 us; error 39 us; skew 500 ppm
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:38.191038 10651 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:38.191493 10651 webserver.cc:460] Webserver started at http://127.7.105.67:40813/ using document root <none> and password file <none>
I20260809 00:37:38.191758 10651 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:38.191807 10651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:38.191978 10651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:38.193092 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "42eb1f836ec14032b636c5fac21ae2e8"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "40ec2ca94a5cc3fff6f597add41b5da7"
server_key_iv: "f7fcf7e0993ad755cac750cb59a5f513"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.193475 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "42eb1f836ec14032b636c5fac21ae2e8"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "40ec2ca94a5cc3fff6f597add41b5da7"
server_key_iv: "f7fcf7e0993ad755cac750cb59a5f513"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.194849 10651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:38.195652 10666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.195868 10651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:38.195981 10651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "42eb1f836ec14032b636c5fac21ae2e8"
format_stamp: "Formatted at 2026-08-09 00:37:38 on dist-test-slave-1zqn"
server_key: "40ec2ca94a5cc3fff6f597add41b5da7"
server_key_iv: "f7fcf7e0993ad755cac750cb59a5f513"
server_key_version: "encryptionkey@0"
I20260809 00:37:38.196066 10651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:38.214993 10651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:38.217377 10651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:38.217588 10651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:38.217830 10651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:38.218158 10651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:38.218215 10651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.218258 10651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:38.218284 10651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:38.224375 10651 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:46383
I20260809 00:37:38.224428 10795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:46383 every 8 connection(s)
I20260809 00:37:38.224743 10651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:38.229259 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10651
I20260809 00:37:38.229338 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:38.229530 7589 external_mini_cluster.cc:1428] Setting key 6ac606836076e9d5dcdfbd87fe31778d
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:38.234225 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:38.225477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:49769 (local address 127.7.105.126:36451)
0809 00:37:38.225570 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:38.225573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:38.225738 (+ 165us) server_negotiation.cc:407] Connection header received
0809 00:37:38.226217 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:38.226218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:38.226228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:38.226285 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:38.227232 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.227730 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.228747 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.228931 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.228987 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:38.231659 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:38.231660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:38.231660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:38.231667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:38.232909 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:38.233341 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:38.233342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:38.233342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:38.233393 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:38.233650 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:38.233650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:38.233650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:38.233754 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:38.233803 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:38.234093 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 00:37:38.234158 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:37:38.234403 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:38.225421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36451 (local address 127.7.105.67:49769)
0809 00:37:38.225609 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:38.225647 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:38.226134 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:38.226362 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:38.226365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:38.226938 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:38.227150 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:38.227155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.227811 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.227812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:38.228670 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:38.228673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:38.229008 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:38.229009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:38.229164 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:38.229777 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:38.229789 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:38.231586 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:38.233004 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:38.233011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:38.233013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:38.233262 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:38.233460 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:38.233461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:38.233461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:38.233575 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:38.233818 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:38.233824 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:38.233949 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 00:37:38.234045 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 00:37:38.234177 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260809 00:37:38.235074 10796 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36451
I20260809 00:37:38.235201 10796 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:38.235402 10796 heartbeater.cc:511] Master 127.7.105.126:36451 requested a full tablet report, sending...
I20260809 00:37:38.235836 10286 ts_manager.cc:194] Registered new tserver with Master: 42eb1f836ec14032b636c5fac21ae2e8 (127.7.105.67:46383)
I20260809 00:37:38.236295 10286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:49769
I20260809 00:37:38.244324 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:37:38.258726 10330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:38.257697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.7.105.126:36451)
0809 00:37:38.257774 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:38.257776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:38.257853 (+ 77us) server_negotiation.cc:407] Connection header received
0809 00:37:38.257923 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:38.257924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:38.257933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:38.257990 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:38.258614 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60626: BlockingRecv error: recv got EOF from 127.0.0.1:60626 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:37:38.259464 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10353
I20260809 00:37:38.271387 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10504
I20260809 00:37:38.278885 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10651
I20260809 00:37:38.286211 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10248
2026-08-09T00:37:38Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1945 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10809](info): set up 2 sockets
Aug 09 00:37:38 dist-test-slave-1zqn krb5kdc[10809](info): commencing operation
krb5kdc: starting...
W20260809 00:37:39.406199 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:39.543814 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:35799
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:35799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:39.630039 10823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:39.630270 10823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:39.630292 10823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:39.631477 10823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:39.631522 10823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:39.631537 10823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:39.631551 10823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:39.631565 10823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:39.633212 10823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:35799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:39.633486 10823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:39.633736 10823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:39.638890 10832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:39.638922 10829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:39.638890 10828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:39.639272 10823 server_base.cc:1034] running on GCE node
I20260809 00:37:39.639564 10823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:39.639859 10823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:39.641004 10823 hybrid_clock.cc:648] HybridClock initialized: now 1786235859640983 us; error 32 us; skew 500 ppm
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:39.648991 10823 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:39.649653 10823 webserver.cc:460] Webserver started at http://127.7.105.126:44169/ using document root <none> and password file <none>
I20260809 00:37:39.649924 10823 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:39.650005 10823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:39.650187 10823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:39.651429 10823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "2e0bfae24b754c4ab2255ccddf146fd1"
format_stamp: "Formatted at 2026-08-09 00:37:39 on dist-test-slave-1zqn"
server_key: "38996b0b5b59f0a574539687311a77e9"
server_key_iv: "a3d2c19a179b67ce9f9451720bd85d45"
server_key_version: "encryptionkey@0"
I20260809 00:37:39.651863 10823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "2e0bfae24b754c4ab2255ccddf146fd1"
format_stamp: "Formatted at 2026-08-09 00:37:39 on dist-test-slave-1zqn"
server_key: "38996b0b5b59f0a574539687311a77e9"
server_key_iv: "a3d2c19a179b67ce9f9451720bd85d45"
server_key_version: "encryptionkey@0"
I20260809 00:37:39.653601 10823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:39.654548 10840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:39.654805 10823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:39.654961 10823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "2e0bfae24b754c4ab2255ccddf146fd1"
format_stamp: "Formatted at 2026-08-09 00:37:39 on dist-test-slave-1zqn"
server_key: "38996b0b5b59f0a574539687311a77e9"
server_key_iv: "a3d2c19a179b67ce9f9451720bd85d45"
server_key_version: "encryptionkey@0"
I20260809 00:37:39.655059 10823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:39.675763 10823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:39.678067 10823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:39.678265 10823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:39.683075 10823 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:35799
I20260809 00:37:39.683112 10897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:35799 every 8 connection(s)
I20260809 00:37:39.683477 10823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:39.684407 10898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:39.686789 10898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1: Bootstrap starting.
I20260809 00:37:39.687445 10898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:39.687773 10898 log.cc:824] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:39.688730 10898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1: No bootstrap required, opened a new log
I20260809 00:37:39.690025 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10823
I20260809 00:37:39.690156 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:39.690343 7589 external_mini_cluster.cc:1428] Setting key 12b341217173da8f5e79bcad1b305dc3
I20260809 00:37:39.690464 10898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } }
I20260809 00:37:39.690689 10898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:39.690737 10898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0bfae24b754c4ab2255ccddf146fd1, State: Initialized, Role: FOLLOWER
I20260809 00:37:39.690855 10898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } }
I20260809 00:37:39.690995 10898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:39.691051 10898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:39.691107 10898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:39.691706 10898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } }
I20260809 00:37:39.691855 10898 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e0bfae24b754c4ab2255ccddf146fd1; no voters:
I20260809 00:37:39.692090 10898 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:39.692142 10904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:39.692283 10904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 2e0bfae24b754c4ab2255ccddf146fd1, State: Running, Role: LEADER
I20260809 00:37:39.692384 10898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:39.692418 10904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } }
I20260809 00:37:39.693156 10905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e0bfae24b754c4ab2255ccddf146fd1. Latest consensus state: current_term: 1 leader_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } } }
I20260809 00:37:39.693267 10905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:39.693404 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0bfae24b754c4ab2255ccddf146fd1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35799 } } }
I20260809 00:37:39.693449 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:39.693634 10913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:39.694309 10913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:39.696254 10913 catalog_manager.cc:1347] Generated new cluster ID: 7c3773d8e90f46b2964a619a3ac11ea2
I20260809 00:37:39.696309 10913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:39.702100 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:39.690885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.7.105.126:35799)
0809 00:37:39.691458 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:39.691466 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:39.691493 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:37:39.691589 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:39.691595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:39.691645 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:39.691760 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:39.692655 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:39.694216 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:39.695084 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:39.695672 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:39.695789 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:39.698254 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:39.698258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:39.698259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:39.698274 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:39.700046 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:39.700485 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:39.700489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:39.700489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:39.700943 (+ 454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:39.701178 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:39.701179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:39.701179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:39.701436 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:39.701628 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:39.701699 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 00:37:39.701850 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":482,"threads_started":1}
I20260809 00:37:39.702171 10913 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:39.702926 10913 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:39.703266 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:39.716609 10913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e0bfae24b754c4ab2255ccddf146fd1: Generated new TSK 0
I20260809 00:37:39.716841 10913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:39.726449 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:39.721663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.7.105.126:35799)
0809 00:37:39.721715 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:39.721718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:39.721952 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:37:39.722012 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:39.722014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:39.722033 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:39.722101 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:39.722586 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:39.723386 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:39.724646 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:39.724983 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:39.725067 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:39.725880 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:39.725882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:39.725882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:39.726148 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:39.726168 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:39.726177 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:39.726241 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:39.726332 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:37:39.726361 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:39.729195 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10823
2026-08-09T00:37:39Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1446 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10936](info): set up 2 sockets
Aug 09 00:37:39 dist-test-slave-1zqn krb5kdc[10936](info): commencing operation
krb5kdc: starting...
W20260809 00:37:40.833700 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:40 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:40.967207 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36747
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:41.051079 10950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:41.051285 10950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:41.051306 10950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:41.052551 10950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:41.052596 10950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:41.052611 10950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:41.052624 10950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:41.052637 10950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:41.054221 10950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:41.054476 10950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:41.054729 10950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:41.059795 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:41.059806 10956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.060039 10950 server_base.cc:1034] running on GCE node
W20260809 00:37:41.059830 10955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.060439 10950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:41.060703 10950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:41.061852 10950 hybrid_clock.cc:648] HybridClock initialized: now 1786235861061827 us; error 41 us; skew 500 ppm
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:41.064460 10950 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:41.064951 10950 webserver.cc:460] Webserver started at http://127.7.105.126:35585/ using document root <none> and password file <none>
I20260809 00:37:41.065213 10950 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:41.065266 10950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:41.065423 10950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:41.066530 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "cd1cb3c3c19c4f339dc2f267f015c938"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "32c187abf6ed701c13ab4dc186535b50"
server_key_iv: "9124c4a884eb3899fda8f6aeae277bfc"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.066906 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "cd1cb3c3c19c4f339dc2f267f015c938"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "32c187abf6ed701c13ab4dc186535b50"
server_key_iv: "9124c4a884eb3899fda8f6aeae277bfc"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.068382 10950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:41.069344 10966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.069568 10950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:41.069696 10950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "cd1cb3c3c19c4f339dc2f267f015c938"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "32c187abf6ed701c13ab4dc186535b50"
server_key_iv: "9124c4a884eb3899fda8f6aeae277bfc"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.069785 10950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:41.076998 10950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:41.079154 10950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:41.079337 10950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:41.084161 10950 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:36747
I20260809 00:37:41.084233 11022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:36747 every 8 connection(s)
I20260809 00:37:41.084690 10950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:41.085815 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:41.088227 11023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938: Bootstrap starting.
I20260809 00:37:41.088845 11023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:41.089102 11023 log.cc:824] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:41.089815 11023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938: No bootstrap required, opened a new log
I20260809 00:37:41.091409 11023 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } }
I20260809 00:37:41.091584 11023 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:41.091611 11023 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1cb3c3c19c4f339dc2f267f015c938, State: Initialized, Role: FOLLOWER
I20260809 00:37:41.091782 11023 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } }
I20260809 00:37:41.091876 11023 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:41.091907 11023 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:41.091938 11023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:41.092813 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 10950
I20260809 00:37:41.092813 11023 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } }
I20260809 00:37:41.092923 11023 leader_election.cc:302] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd1cb3c3c19c4f339dc2f267f015c938; no voters:
I20260809 00:37:41.092937 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:41.093096 11023 leader_election.cc:288] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:41.093155 7589 external_mini_cluster.cc:1428] Setting key 18ebad81dcc75a36398167ebac79717a
I20260809 00:37:41.093158 11026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:41.093309 11026 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [term 1 LEADER]: Becoming Leader. State: Replica: cd1cb3c3c19c4f339dc2f267f015c938, State: Running, Role: LEADER
I20260809 00:37:41.093391 11023 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:41.093405 11026 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } }
I20260809 00:37:41.093875 11027 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } } }
I20260809 00:37:41.093978 11027 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:41.094157 11028 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd1cb3c3c19c4f339dc2f267f015c938. Latest consensus state: current_term: 1 leader_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1cb3c3c19c4f339dc2f267f015c938" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36747 } } }
I20260809 00:37:41.094246 11028 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:41.094296 11036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:41.094878 11036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:41.096855 11036 catalog_manager.cc:1347] Generated new cluster ID: 3c3c98183494443ba0d4a5614fee0771
I20260809 00:37:41.096913 11036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:41.106715 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.093890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.7.105.126:36747)
0809 00:37:41.094097 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:41.094109 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:41.094145 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:37:41.094276 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:41.094284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:41.094345 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:41.095154 (+ 809us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:41.095314 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.096792 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.097938 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.098578 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.098721 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.101803 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:41.101810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:41.101810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:41.101824 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:41.104850 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.105286 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.105290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.105290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.105434 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.105649 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.105650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.105651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.106029 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:41.106206 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:41.106289 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:37:41.106446 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260809 00:37:41.116854 11036 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:41.117434 11036 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:41.120271 11036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd1cb3c3c19c4f339dc2f267f015c938: Generated new TSK 0
I20260809 00:37:41.120474 11036 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:41.155797 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:41.253541 11051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:41.253794 11051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:41.253836 11051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:41.254997 11051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:41.255079 11051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:41.255127 11051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:41.256752 11051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:41.257016 11051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:41.257256 11051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:41.262626 11061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:41.262662 11057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:41.262627 11058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.262893 11051 server_base.cc:1034] running on GCE node
I20260809 00:37:41.263114 11051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:41.263370 11051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:41.264572 11051 hybrid_clock.cc:648] HybridClock initialized: now 1786235861264574 us; error 49 us; skew 500 ppm
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:41.267623 11051 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:41.268253 11051 webserver.cc:460] Webserver started at http://127.7.105.65:35445/ using document root <none> and password file <none>
I20260809 00:37:41.268534 11051 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:41.268620 11051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:41.268801 11051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:41.270027 11051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "8e754ea55d3548a197364a2ebbe466ec"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "371f0f562a32eebbdacf8d6d2171eda4"
server_key_iv: "b6d86ff186317d6537fa52fa7e636226"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.270462 11051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "8e754ea55d3548a197364a2ebbe466ec"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "371f0f562a32eebbdacf8d6d2171eda4"
server_key_iv: "b6d86ff186317d6537fa52fa7e636226"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.272115 11051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 00:37:41.273044 11068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.273239 11051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:41.273353 11051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "8e754ea55d3548a197364a2ebbe466ec"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "371f0f562a32eebbdacf8d6d2171eda4"
server_key_iv: "b6d86ff186317d6537fa52fa7e636226"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.273433 11051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:41.306080 11051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:41.308650 11051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:41.308828 11051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:41.309106 11051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:41.309444 11051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:41.309500 11051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.309556 11051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:41.309578 11051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.315781 11051 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:36997
I20260809 00:37:41.315836 11200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:36997 every 8 connection(s)
I20260809 00:37:41.316190 11051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:41.322934 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11051
I20260809 00:37:41.323056 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:41.323259 7589 external_mini_cluster.cc:1428] Setting key 1d35257c0018c491f0e5a7470b5bc78e
I20260809 00:37:41.326381 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.317031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:40083 (local address 127.7.105.126:36747)
0809 00:37:41.317143 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:41.317145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:41.317346 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:37:41.317908 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:41.317909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:41.317927 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:41.317996 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:41.319041 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.319567 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.320627 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.320851 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.320938 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.323039 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:41.323041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:41.323041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:41.323049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:41.324907 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.325320 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.325321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.325322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.325389 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.325638 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.325639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.325640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.325792 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:41.325864 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:41.326243 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 00:37:41.326315 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 00:37:41.326689 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.316930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36747 (local address 127.7.105.65:40083)
0809 00:37:41.317184 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:41.317237 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:41.317783 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:41.318073 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:41.318081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:41.318717 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:41.318918 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.318923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.319675 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.319677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.320520 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.320522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.320947 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.320948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.321081 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.321540 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:41.321549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:41.322932 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:41.325008 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.325017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.325019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.325241 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.325469 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.325470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.325470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.325572 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.325887 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:41.325895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:41.326079 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 00:37:41.326203 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 00:37:41.326385 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":124,"threads_started":1}
I20260809 00:37:41.327211 11201 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36747
I20260809 00:37:41.327342 11201 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:41.327564 11201 heartbeater.cc:511] Master 127.7.105.126:36747 requested a full tablet report, sending...
I20260809 00:37:41.328197 10984 ts_manager.cc:194] Registered new tserver with Master: 8e754ea55d3548a197364a2ebbe466ec (127.7.105.65:36997)
I20260809 00:37:41.328828 10984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:40083
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:41.355368 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:41.439014 11206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:41.439244 11206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:41.439267 11206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:41.440395 11206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:41.440430 11206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:41.440456 11206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:41.442018 11206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:41.442222 11206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:41.442400 11206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:41.447732 11212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:41.447726 11211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.447781 11206 server_base.cc:1034] running on GCE node
W20260809 00:37:41.447726 11214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.448303 11206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:41.448567 11206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:41.449740 11206 hybrid_clock.cc:648] HybridClock initialized: now 1786235861449713 us; error 44 us; skew 500 ppm
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:41.452337 11206 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:41.452768 11206 webserver.cc:460] Webserver started at http://127.7.105.66:39717/ using document root <none> and password file <none>
I20260809 00:37:41.453011 11206 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:41.453090 11206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:41.453238 11206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:41.454556 11206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "ec873706ec1a4ff5a6c43ba8e71b533a"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "1e16f9d1a01685a99807abe4c3daf4d6"
server_key_iv: "cfa9efae3fd8f07fb75f16b88d7de0b7"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.454943 11206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "ec873706ec1a4ff5a6c43ba8e71b533a"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "1e16f9d1a01685a99807abe4c3daf4d6"
server_key_iv: "cfa9efae3fd8f07fb75f16b88d7de0b7"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.456387 11206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 00:37:41.457197 11223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.457365 11206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:41.457472 11206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "ec873706ec1a4ff5a6c43ba8e71b533a"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "1e16f9d1a01685a99807abe4c3daf4d6"
server_key_iv: "cfa9efae3fd8f07fb75f16b88d7de0b7"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.457582 11206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:41.491067 11206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:41.493614 11206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:41.493817 11206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:41.494117 11206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:41.494450 11206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:41.494509 11206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.494562 11206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:41.494618 11206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.500633 11206 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:34565
I20260809 00:37:41.500679 11356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:34565 every 8 connection(s)
I20260809 00:37:41.501068 11206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:41.501662 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11206
I20260809 00:37:41.501771 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:41.501968 7589 external_mini_cluster.cc:1428] Setting key 343cd3fb8a3caf83b22d81cee9f0defc
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
I20260809 00:37:41.513377 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.501982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41051 (local address 127.7.105.126:36747)
0809 00:37:41.502067 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:41.502069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:41.502320 (+ 251us) server_negotiation.cc:407] Connection header received
0809 00:37:41.502986 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:41.502986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:41.502997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:41.503042 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:41.504471 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.505072 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.506625 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.506972 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.507059 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.510066 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:41.510068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:41.510068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:41.510077 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:41.511776 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.512315 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.512317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.512317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.512385 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.512674 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.512675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.512675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.512799 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:41.512887 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:41.513190 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:37:41.513283 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:37:41.513588 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.501947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36747 (local address 127.7.105.66:41051)
0809 00:37:41.502155 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:41.502211 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:41.502869 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:41.503128 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:41.503132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:41.504005 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:41.504351 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.504359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.505206 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.505208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.506466 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.506470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.507072 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.507073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.507189 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.507831 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:41.507841 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:41.509951 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:41.511881 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.511888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.511890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.512244 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.512473 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.512474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.512474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.512593 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.512876 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:41.512884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:41.513041 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 00:37:41.513157 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:37:41.513319 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260809 00:37:41.514194 11357 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36747
I20260809 00:37:41.514314 11357 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:41.514595 11357 heartbeater.cc:511] Master 127.7.105.126:36747 requested a full tablet report, sending...
I20260809 00:37:41.515069 10984 ts_manager.cc:194] Registered new tserver with Master: ec873706ec1a4ff5a6c43ba8e71b533a (127.7.105.66:34565)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
I20260809 00:37:41.515645 10984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:41051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:41.531533 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:41.616734 11362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:41.616948 11362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:41.616974 11362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:41.618093 11362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:41.618127 11362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:41.618153 11362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:41.619571 11362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:36747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:41.619756 11362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:41.619935 11362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:41.625403 11370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.625510 11362 server_base.cc:1034] running on GCE node
W20260809 00:37:41.625403 11368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:41.625414 11367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:41.625932 11362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:41.626192 11362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:41.627360 11362 hybrid_clock.cc:648] HybridClock initialized: now 1786235861627347 us; error 46 us; skew 500 ppm
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:41.629918 11362 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:41.630402 11362 webserver.cc:460] Webserver started at http://127.7.105.67:42765/ using document root <none> and password file <none>
I20260809 00:37:41.630643 11362 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:41.630692 11362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:41.630828 11362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:41.631889 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "71358260789a41d5a3fa7c7e0dcc52ef"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "d55e1af258adca6c9a6130f8c5e74af0"
server_key_iv: "fd8e9756aa1f693e2b7cdfef250ca6b1"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.632304 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "71358260789a41d5a3fa7c7e0dcc52ef"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "d55e1af258adca6c9a6130f8c5e74af0"
server_key_iv: "fd8e9756aa1f693e2b7cdfef250ca6b1"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.633708 11362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:41.634478 11377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.634693 11362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:41.634788 11362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "71358260789a41d5a3fa7c7e0dcc52ef"
format_stamp: "Formatted at 2026-08-09 00:37:41 on dist-test-slave-1zqn"
server_key: "d55e1af258adca6c9a6130f8c5e74af0"
server_key_iv: "fd8e9756aa1f693e2b7cdfef250ca6b1"
server_key_version: "encryptionkey@0"
I20260809 00:37:41.634837 11362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:41.653991 11362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:41.656265 11362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:41.656435 11362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:41.656697 11362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:41.656992 11362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:41.657025 11362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.657051 11362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:41.657069 11362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:41.663033 11362 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:32829
I20260809 00:37:41.663121 11505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:32829 every 8 connection(s)
I20260809 00:37:41.663460 11362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:41.667256 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11362
I20260809 00:37:41.667356 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:41.667532 7589 external_mini_cluster.cc:1428] Setting key ff7430d87287e046b04b1ad2efcd60da
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:41.673568 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.664347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:43359 (local address 127.7.105.126:36747)
0809 00:37:41.664417 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:41.664419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:41.664618 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:37:41.665186 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:41.665188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:41.665202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:41.665254 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:41.666228 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.666736 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.668195 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.668467 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.668597 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.670791 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:41.670793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:41.670794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:41.670801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:41.672049 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.672530 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.672531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.672531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.672605 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.672869 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.672870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.672871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.672998 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:41.673078 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:41.673372 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 00:37:41.673456 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:41.673751 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.664337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36747 (local address 127.7.105.67:43359)
0809 00:37:41.664493 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:41.664530 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:41.665046 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:41.665325 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:41.665329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:41.665910 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:41.666111 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.666116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.666842 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.666843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.668015 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:41.668018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.668638 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.668639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:41.668813 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.669347 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:41.669356 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:41.670690 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:41.672174 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.672181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.672183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.672414 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.672685 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:41.672686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:41.672686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:41.672796 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:41.673089 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:41.673096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:41.673231 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 00:37:41.673349 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:37:41.673514 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260809 00:37:41.674269 11506 heartbeater.cc:347] Connected to a master server at 127.7.105.126:36747
I20260809 00:37:41.674392 11506 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:41.674618 11506 heartbeater.cc:511] Master 127.7.105.126:36747 requested a full tablet report, sending...
I20260809 00:37:41.675014 10984 ts_manager.cc:194] Registered new tserver with Master: 71358260789a41d5a3fa7c7e0dcc52ef (127.7.105.67:32829)
I20260809 00:37:41.675427 10984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:43359
I20260809 00:37:41.679486 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[10936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235861, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:41.706059 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:41.697310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.7.105.126:36747)
0809 00:37:41.697409 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:41.697411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:41.697518 (+ 107us) server_negotiation.cc:407] Connection header received
0809 00:37:41.697612 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:41.697614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:41.697625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:41.697681 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:41.698338 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.698870 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.700209 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:41.700417 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:41.700503 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:41.703154 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:41.703156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:41.703157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:41.703166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:41.704640 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.705083 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.705084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.705085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.705153 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:41.705401 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:41.705402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:41.705402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:41.705587 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:41.705710 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:41.705841 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:37:41.705938 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 00:37:41.706444 10984 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48082
I20260809 00:37:41.706966 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11051
I20260809 00:37:41.717693 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11206
I20260809 00:37:41.724941 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11362
I20260809 00:37:41.731364 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 10950
2026-08-09T00:37:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1998 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[11519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[11519](info): set up 2 sockets
Aug 09 00:37:41 dist-test-slave-1zqn krb5kdc[11519](info): commencing operation
krb5kdc: starting...
W20260809 00:37:42.856232 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:42 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:42.987943 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46021
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:46021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:43.074118 11532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:43.074337 11532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:43.074360 11532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:43.075470 11532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:43.075506 11532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:43.075520 11532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:43.075532 11532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:43.075544 11532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:43.077159 11532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:46021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:43.077383 11532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:43.077576 11532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:43.082952 11540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.083122 11532 server_base.cc:1034] running on GCE node
W20260809 00:37:43.082957 11539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.082959 11542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.083534 11532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:43.083840 11532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:43.085002 11532 hybrid_clock.cc:648] HybridClock initialized: now 1786235863084971 us; error 49 us; skew 500 ppm
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:43.087575 11532 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:43.088171 11532 webserver.cc:460] Webserver started at http://127.7.105.126:46551/ using document root <none> and password file <none>
I20260809 00:37:43.088438 11532 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:43.088522 11532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:43.088683 11532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:43.089892 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "24a813d564ec445eb494a4651a933d73"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "b8fa5436e4fa7d1c66af897b1df29785"
server_key_iv: "340aa40209d694f3b894450a38238fc9"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.090276 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "24a813d564ec445eb494a4651a933d73"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "b8fa5436e4fa7d1c66af897b1df29785"
server_key_iv: "340aa40209d694f3b894450a38238fc9"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.091702 11532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:43.092633 11549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.092840 11532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:43.092962 11532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "24a813d564ec445eb494a4651a933d73"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "b8fa5436e4fa7d1c66af897b1df29785"
server_key_iv: "340aa40209d694f3b894450a38238fc9"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.093045 11532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:43.108326 11532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:43.110805 11532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:43.111020 11532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:43.115870 11532 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:46021
I20260809 00:37:43.115899 11613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:46021 every 8 connection(s)
I20260809 00:37:43.116592 11532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:43.117311 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:43.119798 11614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73: Bootstrap starting.
I20260809 00:37:43.120477 11614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:43.120808 11614 log.cc:824] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:43.121572 11614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73: No bootstrap required, opened a new log
I20260809 00:37:43.123178 11614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } }
I20260809 00:37:43.123317 11614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:43.123376 11614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a813d564ec445eb494a4651a933d73, State: Initialized, Role: FOLLOWER
I20260809 00:37:43.123526 11614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } }
I20260809 00:37:43.123623 11614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:43.123687 11614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:43.123730 11614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:43.123888 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11532
I20260809 00:37:43.123997 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:43.124229 7589 external_mini_cluster.cc:1428] Setting key 92d07e1cced057364c85a35137d8bdaf
I20260809 00:37:43.125065 11614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } }
I20260809 00:37:43.125178 11614 leader_election.cc:302] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a813d564ec445eb494a4651a933d73; no voters:
I20260809 00:37:43.125339 11614 leader_election.cc:288] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:43.125399 11619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:43.125679 11619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [term 1 LEADER]: Becoming Leader. State: Replica: 24a813d564ec445eb494a4651a933d73, State: Running, Role: LEADER
I20260809 00:37:43.125715 11614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:43.125787 11619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } }
I20260809 00:37:43.126143 11619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a813d564ec445eb494a4651a933d73. Latest consensus state: current_term: 1 leader_uuid: "24a813d564ec445eb494a4651a933d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } } }
I20260809 00:37:43.126241 11619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:43.126154 11620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a813d564ec445eb494a4651a933d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a813d564ec445eb494a4651a933d73" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46021 } } }
I20260809 00:37:43.126442 11620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:43.126458 11629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:43.127161 11629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:43.129127 11629 catalog_manager.cc:1347] Generated new cluster ID: cb6b7300c0f14f40ab88c724098e98dc
I20260809 00:37:43.129180 11629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:43.137249 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.124794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.7.105.126:46021)
0809 00:37:43.125015 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.125025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.125077 (+ 52us) server_negotiation.cc:407] Connection header received
0809 00:37:43.125199 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.125206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.125267 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.125426 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.126305 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.127862 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.128813 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.129555 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.129683 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.132918 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.132922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.132923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.132936 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.135043 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.135475 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.135478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.135479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.135635 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.135873 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.135874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.135874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.136243 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.136403 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.136874 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.137037 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260809 00:37:43.141062 11629 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:43.141769 11629 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:43.159773 11629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a813d564ec445eb494a4651a933d73: Generated new TSK 0
I20260809 00:37:43.160027 11629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:43.193181 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:43.300638 11644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:43.300873 11644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:43.300915 11644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:43.302114 11644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:43.302179 11644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:43.302223 11644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:43.303781 11644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:43.304085 11644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:43.304338 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:43.309538 11649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.309541 11650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.309538 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.309832 11644 server_base.cc:1034] running on GCE node
I20260809 00:37:43.310120 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:43.310333 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:43.311488 11644 hybrid_clock.cc:648] HybridClock initialized: now 1786235863311473 us; error 28 us; skew 500 ppm
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:43.314304 11644 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:43.314706 11644 webserver.cc:460] Webserver started at http://127.7.105.65:41999/ using document root <none> and password file <none>
I20260809 00:37:43.314965 11644 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:43.315012 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:43.315189 11644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:43.316331 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "757a1952a59745d88f7026e4ff2df216"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "06da4d76c7ee9926cff8150012214bd4"
server_key_iv: "58b049b58d479447169485ec5f4a15ba"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.316710 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "757a1952a59745d88f7026e4ff2df216"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "06da4d76c7ee9926cff8150012214bd4"
server_key_iv: "58b049b58d479447169485ec5f4a15ba"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.318112 11644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:43.318897 11659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.319070 11644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:43.319180 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "757a1952a59745d88f7026e4ff2df216"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "06da4d76c7ee9926cff8150012214bd4"
server_key_iv: "58b049b58d479447169485ec5f4a15ba"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.319254 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:43.339309 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:43.341428 11644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:43.341604 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:43.341857 11644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:43.342182 11644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:43.342238 11644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.342284 11644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:43.342316 11644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.347999 11644 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:44725
I20260809 00:37:43.348045 11791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:44725 every 8 connection(s)
I20260809 00:37:43.348414 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:43.350126 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11644
I20260809 00:37:43.350220 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:43.350428 7589 external_mini_cluster.cc:1428] Setting key 2cf0675cedc4b30ce5d23f2a380b61fe
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:43.359979 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.349174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:46021 (local address 127.7.105.65:42305)
0809 00:37:43.349357 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:43.349396 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:43.350440 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:43.350722 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:43.350725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:43.351108 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:43.351302 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.351307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.352045 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.352047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.353078 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.353081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.353537 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.353539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.353694 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.354223 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:43.354234 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:43.355894 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:43.358099 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.358106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.358109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.358424 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.358747 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.358748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.358749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.358889 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.359316 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:43.359324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:43.359583 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 00:37:43.359689 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 00:37:43.359814 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260809 00:37:43.359973 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.349194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:42305 (local address 127.7.105.126:46021)
0809 00:37:43.349266 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.349269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.349488 (+ 219us) server_negotiation.cc:407] Connection header received
0809 00:37:43.350566 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.350568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.350584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.350627 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.351409 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.351949 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.353182 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.353455 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.353510 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.356123 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.356125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.356125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.356137 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.357988 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.358542 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.358543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.358544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.358653 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.358965 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.358966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.358966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.359214 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.359292 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.359749 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.359844 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:37:43.360514 11792 heartbeater.cc:347] Connected to a master server at 127.7.105.126:46021
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:37:43.360634 11792 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:43.360862 11792 heartbeater.cc:511] Master 127.7.105.126:46021 requested a full tablet report, sending...
I20260809 00:37:43.361455 11567 ts_manager.cc:194] Registered new tserver with Master: 757a1952a59745d88f7026e4ff2df216 (127.7.105.65:44725)
I20260809 00:37:43.362107 11567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:42305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:43.378821 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:43.462224 11798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:43.462438 11798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:43.462461 11798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:43.463649 11798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:43.463692 11798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:43.463719 11798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:43.465291 11798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:43.465557 11798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:43.465797 11798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:43.471096 11805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.471174 11807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.471122 11803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.471733 11798 server_base.cc:1034] running on GCE node
I20260809 00:37:43.471944 11798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:43.472249 11798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:43.473398 11798 hybrid_clock.cc:648] HybridClock initialized: now 1786235863473377 us; error 37 us; skew 500 ppm
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:43.475838 11798 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:43.476387 11798 webserver.cc:460] Webserver started at http://127.7.105.66:37025/ using document root <none> and password file <none>
I20260809 00:37:43.476640 11798 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:43.476686 11798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:43.476861 11798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:43.477936 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "7810a7af75dc4ceaafdefdf956e0a568"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "235afd594de2e616953a13eb3bb71f8f"
server_key_iv: "38ddf9233ac1f8dea06c38a7180dd835"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.478300 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "7810a7af75dc4ceaafdefdf956e0a568"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "235afd594de2e616953a13eb3bb71f8f"
server_key_iv: "38ddf9233ac1f8dea06c38a7180dd835"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.479689 11798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:43.480494 11814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.480672 11798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.480782 11798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "7810a7af75dc4ceaafdefdf956e0a568"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "235afd594de2e616953a13eb3bb71f8f"
server_key_iv: "38ddf9233ac1f8dea06c38a7180dd835"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.480855 11798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:43.498293 11798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:43.500222 11798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:43.500381 11798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:43.500627 11798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:43.500950 11798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:43.501005 11798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.501060 11798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:43.501109 11798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.507182 11798 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:34509
I20260809 00:37:43.507231 11941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:34509 every 8 connection(s)
I20260809 00:37:43.507581 11798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:43.514237 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11798
I20260809 00:37:43.514367 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:43.514582 7589 external_mini_cluster.cc:1428] Setting key 0970d77367c8cc3cbf1039c1119d35a5
I20260809 00:37:43.517545 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.508422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35897 (local address 127.7.105.126:46021)
0809 00:37:43.508533 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.508535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.508718 (+ 183us) server_negotiation.cc:407] Connection header received
0809 00:37:43.509725 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.509727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.509740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.509805 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.510443 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.510961 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.511976 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.512270 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.512358 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.514727 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.514729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.514729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.514737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.516146 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.516577 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.516579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.516579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.516642 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.516896 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.516897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.516897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.517016 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.517096 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.517365 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.517456 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:37:43.517738 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.508393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:46021 (local address 127.7.105.66:35897)
0809 00:37:43.508589 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:43.508627 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:43.509607 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:43.509883 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:43.509886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:43.510146 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:43.510341 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.510345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.511076 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.511077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.511881 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.511883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.512369 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.512370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.512490 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.513032 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:43.513041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:43.514640 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:43.516236 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.516242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.516245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.516468 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.516710 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.516711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.516711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.516820 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.517087 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:43.517094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:43.517232 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 00:37:43.517343 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:37:43.517499 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260809 00:37:43.518275 11943 heartbeater.cc:347] Connected to a master server at 127.7.105.126:46021
I20260809 00:37:43.518378 11943 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:43.518579 11943 heartbeater.cc:511] Master 127.7.105.126:46021 requested a full tablet report, sending...
I20260809 00:37:43.518950 11567 ts_manager.cc:194] Registered new tserver with Master: 7810a7af75dc4ceaafdefdf956e0a568 (127.7.105.66:34509)
I20260809 00:37:43.519357 11567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:35897
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:43.541241 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:43.624594 11950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:43.624797 11950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:43.624822 11950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:43.626258 11950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:43.626300 11950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:43.626327 11950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:43.628134 11950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:46021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:43.628371 11950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:43.628610 11950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:43.633808 11955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.633893 11950 server_base.cc:1034] running on GCE node
W20260809 00:37:43.633792 11959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:43.633792 11956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:43.634275 11950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:43.634513 11950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:43.635682 11950 hybrid_clock.cc:648] HybridClock initialized: now 1786235863635665 us; error 33 us; skew 500 ppm
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:43.638242 11950 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:43.638706 11950 webserver.cc:460] Webserver started at http://127.7.105.67:46073/ using document root <none> and password file <none>
I20260809 00:37:43.638913 11950 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:43.638953 11950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:43.639045 11950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:43.640018 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "537aec80ad1c423ba60c94aaaec9563d"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "82d1ec037bf019ab901f67588644be32"
server_key_iv: "08a8f9783339f2c4e03d59bac9dc1610"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.640383 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "537aec80ad1c423ba60c94aaaec9563d"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "82d1ec037bf019ab901f67588644be32"
server_key_iv: "08a8f9783339f2c4e03d59bac9dc1610"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.641664 11950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:43.642359 11967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.642526 11950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:43.642621 11950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "537aec80ad1c423ba60c94aaaec9563d"
format_stamp: "Formatted at 2026-08-09 00:37:43 on dist-test-slave-1zqn"
server_key: "82d1ec037bf019ab901f67588644be32"
server_key_iv: "08a8f9783339f2c4e03d59bac9dc1610"
server_key_version: "encryptionkey@0"
I20260809 00:37:43.642675 11950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:43.657151 11950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:43.659456 11950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:43.659626 11950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:43.659870 11950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:43.660267 11950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:43.660303 11950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.660326 11950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:43.660365 11950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:43.666129 11950 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:43585
I20260809 00:37:43.666211 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:43585 every 8 connection(s)
I20260809 00:37:43.666525 11950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:43.676178 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.667310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:57409 (local address 127.7.105.126:46021)
0809 00:37:43.667377 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.667379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.667590 (+ 211us) server_negotiation.cc:407] Connection header received
0809 00:37:43.668638 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.668640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.668654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.668706 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.669331 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.669829 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.670869 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.671070 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.671148 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.673284 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.673285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.673286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.673294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.674655 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.675131 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.675132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.675133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.675198 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.675463 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.675464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.675465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.675593 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.675675 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.675964 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.676077 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:37:43.676400 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.667298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:46021 (local address 127.7.105.67:57409)
0809 00:37:43.667468 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:43.667503 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:43.668513 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:43.668781 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:43.668785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:43.669030 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:43.669221 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.669225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.669931 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.669932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.670777 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:43.670779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.671154 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.671155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:43.671276 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.671718 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:43.671726 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:43.673183 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:43.674772 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.674780 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.674782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.675029 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.675269 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:43.675270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:43.675271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:43.675388 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:43.675666 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:43.675673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:43.675815 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 00:37:43.675940 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 00:37:43.676126 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260809 00:37:43.676548 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 11950
I20260809 00:37:43.676656 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:43.676877 7589 external_mini_cluster.cc:1428] Setting key a8fbc62951da3381ba354d72ac6e9418
I20260809 00:37:43.676908 12100 heartbeater.cc:347] Connected to a master server at 127.7.105.126:46021
I20260809 00:37:43.677016 12100 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:43.677256 12100 heartbeater.cc:511] Master 127.7.105.126:46021 requested a full tablet report, sending...
I20260809 00:37:43.677639 11567 ts_manager.cc:194] Registered new tserver with Master: 537aec80ad1c423ba60c94aaaec9563d (127.7.105.67:43585)
I20260809 00:37:43.677923 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:43.678160 11567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:57409
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:43.703114 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.695217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.7.105.65:44725)
0809 00:37:43.695363 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.695369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.695380 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:43.695415 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.695417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.695431 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.695514 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.696166 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.696910 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.697696 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.698023 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.698107 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.700308 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.700317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.700317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.700324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.701909 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.702265 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.702268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.702269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.702330 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.702533 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.702534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.702534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.702681 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.702853 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.702899 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.702977 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[11519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:43.711328 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:43.704389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.7.105.126:46021)
0809 00:37:43.704456 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:43.704459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:43.704492 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:37:43.704576 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:43.704577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:43.704587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:43.704633 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:43.705122 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.705612 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.706305 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:43.706520 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:43.706614 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:43.708942 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:43.708944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:43.708944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:43.708953 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:43.710335 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.710652 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.710654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.710654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.710712 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:43.710881 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:43.710882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:43.710882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:43.711023 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:43.711107 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:43.711194 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 00:37:43.711260 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 00:37:43.711615 11567 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35808
I20260809 00:37:43.712004 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11644
I20260809 00:37:43.719930 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11798
I20260809 00:37:43.726778 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11950
I20260809 00:37:43.732173 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 11532
2026-08-09T00:37:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2000 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[12115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[12115](info): set up 2 sockets
Aug 09 00:37:43 dist-test-slave-1zqn krb5kdc[12115](info): commencing operation
krb5kdc: starting...
W20260809 00:37:44.862938 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:44 dist-test-slave-1zqn krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:44.995872 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40591
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:45.081151 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:45.081389 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:45.081416 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:45.082567 12130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:45.082607 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:45.082620 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:45.082633 12130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:45.082645 12130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:45.084179 12130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:45.084390 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:45.084581 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:45.089545 12139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.089545 12136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.089552 12137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:45.089810 12130 server_base.cc:1034] running on GCE node
I20260809 00:37:45.090121 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:45.090440 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:45.091603 12130 hybrid_clock.cc:648] HybridClock initialized: now 1786235865091604 us; error 49 us; skew 500 ppm
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:45.097920 12130 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:45.098268 12130 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:45.098315 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:45.098408 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:45.099490 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "0f057d6bf1c44d06a39bde935d234542"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "1897039a6523148a55d220cef9e79485"
server_key_iv: "901f8bfa06f5361b8beff729a3a37ec7"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.099817 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "0f057d6bf1c44d06a39bde935d234542"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "1897039a6523148a55d220cef9e79485"
server_key_iv: "901f8bfa06f5361b8beff729a3a37ec7"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.101367 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:45.102309 12146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.102524 12130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:45.102640 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "0f057d6bf1c44d06a39bde935d234542"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "1897039a6523148a55d220cef9e79485"
server_key_iv: "901f8bfa06f5361b8beff729a3a37ec7"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.102711 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:45.119191 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:45.121855 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:45.121997 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:45.126446 12130 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:40591
I20260809 00:37:45.126508 12210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:40591 every 8 connection(s)
I20260809 00:37:45.126847 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:45.127960 12211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:45.130364 12211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: Bootstrap starting.
I20260809 00:37:45.131008 12211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:45.131335 12211 log.cc:824] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:45.132120 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12130
I20260809 00:37:45.132184 12211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: No bootstrap required, opened a new log
I20260809 00:37:45.132241 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:45.132431 7589 external_mini_cluster.cc:1428] Setting key 32bd29b04f093ea07ff80ae4d3cdbeaf
I20260809 00:37:45.134727 12211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } }
I20260809 00:37:45.134912 12211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:45.134953 12211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f057d6bf1c44d06a39bde935d234542, State: Initialized, Role: FOLLOWER
I20260809 00:37:45.135134 12211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } }
I20260809 00:37:45.135212 12211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:45.135236 12211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:45.135294 12211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:45.135959 12211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } }
I20260809 00:37:45.136097 12211 leader_election.cc:302] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f057d6bf1c44d06a39bde935d234542; no voters:
I20260809 00:37:45.136250 12211 leader_election.cc:288] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:45.136307 12217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:45.136497 12211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:45.136500 12217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [term 1 LEADER]: Becoming Leader. State: Replica: 0f057d6bf1c44d06a39bde935d234542, State: Running, Role: LEADER
I20260809 00:37:45.136708 12217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } }
I20260809 00:37:45.137045 12219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f057d6bf1c44d06a39bde935d234542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } } }
I20260809 00:37:45.137149 12219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:45.137228 12218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f057d6bf1c44d06a39bde935d234542. Latest consensus state: current_term: 1 leader_uuid: "0f057d6bf1c44d06a39bde935d234542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f057d6bf1c44d06a39bde935d234542" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40591 } } }
I20260809 00:37:45.137351 12218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542 [sys.catalog]: This master's current role is: LEADER
W20260809 00:37:45.138494 12235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:45.138571 12235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:45.138688 12227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:45.138868 12227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:45.140692 12227 catalog_manager.cc:1347] Generated new cluster ID: 78222cc1c50645848469a1fd76413690
I20260809 00:37:45.140753 12227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:45.145288 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.132992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.7.105.126:40591)
0809 00:37:45.133230 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.133240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.133275 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:37:45.134671 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.134681 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.134749 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.134911 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:45.135707 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.137150 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.138345 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.138972 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.139088 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.141826 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:45.141830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:45.141830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:45.141843 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:45.143610 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.143993 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.143996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.143996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.144163 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.144377 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.144378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.144379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.144646 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:45.144817 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.144916 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.145077 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":121,"threads_started":1}
I20260809 00:37:45.150131 12227 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:45.150724 12227 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:45.159201 12227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: Generated new TSK 0
I20260809 00:37:45.159441 12227 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:45.193212 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:45.278934 12241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:45.279152 12241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:45.279177 12241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:45.280371 12241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:45.280416 12241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:45.280443 12241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:45.281977 12241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:45.282210 12241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:45.282439 12241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:45.287797 12249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:45.287817 12241 server_base.cc:1034] running on GCE node
W20260809 00:37:45.287799 12246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.288012 12247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:45.288367 12241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:45.288589 12241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:45.289729 12241 hybrid_clock.cc:648] HybridClock initialized: now 1786235865289708 us; error 41 us; skew 500 ppm
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:45.292325 12241 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:45.292644 12241 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:45.292687 12241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:45.292778 12241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:45.293890 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "e363ae19997a472b9097f1f3c9409012"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "58816d8417c150377f871e4078d1bc20"
server_key_iv: "b107b9e5658e98c1f7bf5dd468691273"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.294199 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "e363ae19997a472b9097f1f3c9409012"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "58816d8417c150377f871e4078d1bc20"
server_key_iv: "b107b9e5658e98c1f7bf5dd468691273"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.295456 12241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:45.296164 12255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.296339 12241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:45.296442 12241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "e363ae19997a472b9097f1f3c9409012"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "58816d8417c150377f871e4078d1bc20"
server_key_iv: "b107b9e5658e98c1f7bf5dd468691273"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.296530 12241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:45.322685 12241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:45.325160 12241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:45.325331 12241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:45.325603 12241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:45.325937 12241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:45.325990 12241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.326031 12241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:45.326062 12241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.332088 12241 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:41147
I20260809 00:37:45.332172 12384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:41147 every 8 connection(s)
I20260809 00:37:45.332458 12241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:45.339740 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12241
I20260809 00:37:45.339869 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:45.340140 7589 external_mini_cluster.cc:1428] Setting key 72ab47ae3deb7a1d55ad346a52fb960a
I20260809 00:37:45.342490 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.333222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:38705 (local address 127.7.105.126:40591)
0809 00:37:45.333296 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.333299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.333541 (+ 242us) server_negotiation.cc:407] Connection header received
0809 00:37:45.334012 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.334013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.334065 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.334121 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:45.335072 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.335584 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.336763 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.336958 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.337062 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.339292 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:45.339294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:45.339294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:45.339302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:45.341006 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.341466 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.341467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.341467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.341536 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.341809 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.341810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.341810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.341933 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:45.342010 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.342299 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.342398 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:37:45.342728 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.333207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40591 (local address 127.7.105.65:38705)
0809 00:37:45.333405 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.333446 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.333925 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.334178 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.334182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.334449 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:45.334972 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.334978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.335670 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.335671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.336688 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.336691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.337055 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.337056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.337174 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.337684 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:45.337693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:45.339189 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:45.341130 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.341139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.341142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.341384 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.341607 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.341608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.341608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.341732 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.341999 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:45.342008 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:45.342162 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.342277 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.342444 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260809 00:37:45.343240 12385 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40591
I20260809 00:37:45.343333 12385 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:45.343536 12385 heartbeater.cc:511] Master 127.7.105.126:40591 requested a full tablet report, sending...
I20260809 00:37:45.344193 12168 ts_manager.cc:194] Registered new tserver with Master: e363ae19997a472b9097f1f3c9409012 (127.7.105.65:41147)
I20260809 00:37:45.344775 12168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:38705
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:45.369076 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:45.453709 12390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:45.454057 12390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:45.454108 12390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:45.455441 12390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:45.455518 12390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:45.455569 12390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:45.457360 12390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:45.457675 12390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:45.457954 12390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:45.463842 12396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.463889 12398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.463896 12395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:45.464389 12390 server_base.cc:1034] running on GCE node
I20260809 00:37:45.464597 12390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:45.464815 12390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:45.466046 12390 hybrid_clock.cc:648] HybridClock initialized: now 1786235865466033 us; error 26 us; skew 500 ppm
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:45.468907 12390 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:45.469401 12390 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:45.469458 12390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:45.469650 12390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:45.470822 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "fc71f02621074bcaa130e949835861fd"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "990154d5c6dbf9123e2f3b444ed62989"
server_key_iv: "5eec65c6d88bede0da37993ed0564f11"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.471308 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "fc71f02621074bcaa130e949835861fd"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "990154d5c6dbf9123e2f3b444ed62989"
server_key_iv: "5eec65c6d88bede0da37993ed0564f11"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.472824 12390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:45.473698 12404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.473878 12390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:45.474009 12390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "fc71f02621074bcaa130e949835861fd"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "990154d5c6dbf9123e2f3b444ed62989"
server_key_iv: "5eec65c6d88bede0da37993ed0564f11"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.474097 12390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:45.487826 12390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:45.490485 12390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:45.490664 12390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:45.490922 12390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:45.491283 12390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:45.491344 12390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.491393 12390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:45.491422 12390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.498253 12390 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:43307
I20260809 00:37:45.498296 12533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:43307 every 8 connection(s)
I20260809 00:37:45.498590 12390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:45.504833 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12390
I20260809 00:37:45.504949 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:45.505193 7589 external_mini_cluster.cc:1428] Setting key b32b7effecf1d3381405116e64fc03a3
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:45.508826 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.499353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:36501 (local address 127.7.105.126:40591)
0809 00:37:45.499398 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.499400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.499645 (+ 245us) server_negotiation.cc:407] Connection header received
0809 00:37:45.500226 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.500227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.500238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.500319 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:45.501339 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.501833 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.503015 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.503219 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.503273 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.505825 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:45.505827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:45.505827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:45.505835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:45.507380 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.507800 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.507802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.507802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.507864 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.508151 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.508153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.508153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.508273 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:45.508354 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.508650 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.508732 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:45.509028 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.499358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40591 (local address 127.7.105.66:36501)
0809 00:37:45.499514 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.499553 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.500113 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.500407 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.500412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.500700 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:45.501246 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.501251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.502003 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.502004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.502902 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.502904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.503314 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.503315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.503426 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.503991 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:45.504000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:45.505724 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:45.507467 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.507475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.507478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.507718 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.507936 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.507937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.507937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.508096 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.508344 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:45.508352 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:45.508500 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.508611 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.508783 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
I20260809 00:37:45.514454 12534 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40591
I20260809 00:37:45.514576 12534 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:45.514778 12534 heartbeater.cc:511] Master 127.7.105.126:40591 requested a full tablet report, sending...
I20260809 00:37:45.515242 12168 ts_manager.cc:194] Registered new tserver with Master: fc71f02621074bcaa130e949835861fd (127.7.105.66:43307)
I20260809 00:37:45.515756 12168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:36501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:45.535120 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:45.620607 12541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:45.620816 12541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:45.620841 12541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:45.622040 12541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:45.622085 12541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:45.622114 12541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:45.623776 12541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:45.624043 12541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:45.624312 12541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:45.629458 12546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.629467 12547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:45.629477 12549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:45.629528 12541 server_base.cc:1034] running on GCE node
I20260809 00:37:45.629850 12541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:45.630101 12541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:45.631261 12541 hybrid_clock.cc:648] HybridClock initialized: now 1786235865631256 us; error 37 us; skew 500 ppm
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:45.633809 12541 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:45.634171 12541 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:45.634243 12541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:45.634392 12541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:45.635458 12541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "b706114ceb18421693db4baecd37ff83"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "e749e2822b08b4fe8bba4ddfb3b3aff4"
server_key_iv: "4f573580bd6fdb158b3b7fade1b64f48"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.635819 12541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "b706114ceb18421693db4baecd37ff83"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "e749e2822b08b4fe8bba4ddfb3b3aff4"
server_key_iv: "4f573580bd6fdb158b3b7fade1b64f48"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.637449 12541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:45.638269 12555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.638447 12541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:45.638550 12541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "b706114ceb18421693db4baecd37ff83"
format_stamp: "Formatted at 2026-08-09 00:37:45 on dist-test-slave-1zqn"
server_key: "e749e2822b08b4fe8bba4ddfb3b3aff4"
server_key_iv: "4f573580bd6fdb158b3b7fade1b64f48"
server_key_version: "encryptionkey@0"
I20260809 00:37:45.638655 12541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:45.659668 12541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:45.661765 12541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:45.661931 12541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:45.662182 12541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:45.662495 12541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:45.662546 12541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.662590 12541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:45.662622 12541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:45.668505 12541 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:42963
I20260809 00:37:45.668563 12682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:42963 every 8 connection(s)
I20260809 00:37:45.668865 12541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:45.671231 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12541
I20260809 00:37:45.671315 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:45.671500 7589 external_mini_cluster.cc:1428] Setting key cd63c8a801229ed4a19067f5999985de
Aug 09 00:37:45 dist-test-slave-1zqn krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:45.678593 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.669618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:35915 (local address 127.7.105.126:40591)
0809 00:37:45.669663 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.669665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.669856 (+ 191us) server_negotiation.cc:407] Connection header received
0809 00:37:45.670392 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.670394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.670407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.670472 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:45.671504 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.672322 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.673362 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.673583 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.673660 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.675833 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:45.675834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:45.675834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:45.675841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:45.677435 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.677797 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.677798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.677798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.677849 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.678053 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.678053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.678054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.678160 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:45.678212 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.678445 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.678508 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:45.678826 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.669584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40591 (local address 127.7.105.67:35915)
0809 00:37:45.669733 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.669775 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.670288 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.670544 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.670548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.670828 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:45.671401 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.671408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.672463 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.672468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.673272 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.673274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.673666 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.673668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.673799 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.674290 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:45.674298 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:45.675727 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:45.677505 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.677512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.677514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.677733 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.677904 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:45.677904 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:45.677905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:45.677999 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:45.678213 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:45.678217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:45.678345 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.678426 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.678557 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260809 00:37:45.679369 12683 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40591
I20260809 00:37:45.679473 12683 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:45.679648 12683 heartbeater.cc:511] Master 127.7.105.126:40591 requested a full tablet report, sending...
I20260809 00:37:45.680089 12168 ts_manager.cc:194] Registered new tserver with Master: b706114ceb18421693db4baecd37ff83 (127.7.105.67:42963)
I20260809 00:37:45.680469 12168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:35915
I20260809 00:37:45.683432 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:45.690348 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.684803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.7.105.126:40591)
0809 00:37:45.684864 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.684865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.684959 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:37:45.684990 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.684991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.685007 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.685074 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:45.685636 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.686103 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.686827 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.687003 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.687063 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.688262 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:45.688264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:45.688264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:45.688272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:45.689423 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.689728 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.689729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.689729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.689777 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:45.689937 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:45.689938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:45.689938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:45.690072 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:45.690146 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.690228 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.690285 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:37:45.691908 12168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:45.693604 12168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:45.700575 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.696350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.7.105.65:41147)
0809 00:37:45.696570 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.696574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.696950 (+ 376us) server_negotiation.cc:407] Connection header received
0809 00:37:45.696997 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.697000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.697016 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.697085 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:45.697287 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.697963 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.699335 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700151 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700230 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700289 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.700437 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.700485 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260809 00:37:45.700569 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.696215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41147 (local address 127.0.0.1:56524)
0809 00:37:45.696829 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.696840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.696891 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.697172 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.697172 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.697174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:45.697213 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.697214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.698090 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.698091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.699236 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.699239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700244 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.700322 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700336 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.700409 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.700468 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":33,"thread_start_us":84,"threads_started":1}
I20260809 00:37:45.700656 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.696083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.7.105.66:43307)
0809 00:37:45.696300 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.696305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.696475 (+ 170us) server_negotiation.cc:407] Connection header received
0809 00:37:45.696604 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.696608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.696627 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.696691 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:45.697078 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.698118 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.699469 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700167 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700230 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700290 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.700515 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.700565 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260809 00:37:45.700865 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.695955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43307 (local address 127.0.0.1:52896)
0809 00:37:45.696369 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.696398 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.696480 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.696873 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.696874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.696876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:45.696989 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.696991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.698220 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.698220 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.699383 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.699386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700327 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.700413 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700419 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.700755 (+ 336us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.700798 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":31,"thread_start_us":132,"threads_started":1}
I20260809 00:37:45.700933 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.696087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.7.105.67:42963)
0809 00:37:45.696300 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.696305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.696570 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:37:45.696610 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.696612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.696625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.696709 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:45.696879 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.698017 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.699511 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700233 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700300 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700357 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.700613 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.700845 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":102,"threads_started":1}
I20260809 00:37:45.701097 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.695955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42963 (local address 127.0.0.1:39360)
0809 00:37:45.696364 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.696402 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.696472 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.696716 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.696719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.696721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:45.696785 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.696787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.698140 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.698141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.699429 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.699431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.700390 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.700391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.700456 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.700458 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.700505 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.700535 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":92,"thread_start_us":56,"threads_started":1}
I20260809 00:37:45.701560 12312 tablet_service.cc:1467] Processing CreateTablet for tablet 15f57a1016804e8483ef25fba95b6de2 (DEFAULT_TABLE table=test-table [id=3faf019e58034cfbb809c65d58e7b9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:45.701622 12461 tablet_service.cc:1467] Processing CreateTablet for tablet 15f57a1016804e8483ef25fba95b6de2 (DEFAULT_TABLE table=test-table [id=3faf019e58034cfbb809c65d58e7b9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:45.701733 12605 tablet_service.cc:1467] Processing CreateTablet for tablet 15f57a1016804e8483ef25fba95b6de2 (DEFAULT_TABLE table=test-table [id=3faf019e58034cfbb809c65d58e7b9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:45.701854 12312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f57a1016804e8483ef25fba95b6de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:45.701967 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f57a1016804e8483ef25fba95b6de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:45.701999 12605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f57a1016804e8483ef25fba95b6de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:45.704591 12704 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Bootstrap starting.
I20260809 00:37:45.705216 12705 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Bootstrap starting.
I20260809 00:37:45.705440 12704 tablet_bootstrap.cc:654] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:45.705978 12705 tablet_bootstrap.cc:654] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:45.706126 12704 log.cc:824] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:45.706612 12706 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Bootstrap starting.
I20260809 00:37:45.706612 12705 log.cc:824] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:45.707087 12704 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: No bootstrap required, opened a new log
I20260809 00:37:45.707160 12704 ts_tablet_manager.cc:1397] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:37:45.707324 12706 tablet_bootstrap.cc:654] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:45.707574 12705 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: No bootstrap required, opened a new log
I20260809 00:37:45.707659 12705 ts_tablet_manager.cc:1397] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:37:45.707955 12706 log.cc:824] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:45.708798 12706 tablet_bootstrap.cc:492] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: No bootstrap required, opened a new log
I20260809 00:37:45.708870 12706 ts_tablet_manager.cc:1397] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:45.709069 12704 raft_consensus.cc:348] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.709218 12704 raft_consensus.cc:374] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:45.709263 12704 raft_consensus.cc:729] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e363ae19997a472b9097f1f3c9409012, State: Initialized, Role: FOLLOWER
I20260809 00:37:45.709383 12704 consensus_queue.cc:260] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.709712 12385 heartbeater.cc:503] Master 127.7.105.126:40591 was elected leader, sending a full tablet report...
I20260809 00:37:45.709719 12704 ts_tablet_manager.cc:1428] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:45.709791 12705 raft_consensus.cc:348] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.709936 12705 raft_consensus.cc:374] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:45.709968 12705 raft_consensus.cc:729] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71f02621074bcaa130e949835861fd, State: Initialized, Role: FOLLOWER
I20260809 00:37:45.710119 12705 consensus_queue.cc:260] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.710418 12534 heartbeater.cc:503] Master 127.7.105.126:40591 was elected leader, sending a full tablet report...
I20260809 00:37:45.710747 12705 ts_tablet_manager.cc:1428] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:45.710979 12706 raft_consensus.cc:348] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.711140 12706 raft_consensus.cc:374] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:45.711182 12706 raft_consensus.cc:729] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b706114ceb18421693db4baecd37ff83, State: Initialized, Role: FOLLOWER
I20260809 00:37:45.711292 12706 consensus_queue.cc:260] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.711577 12706 ts_tablet_manager.cc:1428] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:45.711586 12683 heartbeater.cc:503] Master 127.7.105.126:40591 was elected leader, sending a full tablet report...
W20260809 00:37:45.749145 12536 tablet.cc:2369] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:37:45.833221 12386 tablet.cc:2369] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:45.910215 12711 raft_consensus.cc:482] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:45.910408 12711 raft_consensus.cc:504] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.910818 12711 leader_election.cc:288] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b706114ceb18421693db4baecd37ff83 (127.7.105.67:42963), fc71f02621074bcaa130e949835861fd (127.7.105.66:43307)
I20260809 00:37:45.914287 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.911078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42963 (local address 127.7.105.65:49877)
0809 00:37:45.911301 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.911311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.911410 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.911632 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.911633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.911637 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:45.911706 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.911708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.912492 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.912493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.913323 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.913325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.913989 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.913990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.914067 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.914072 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.914126 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.914164 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260809 00:37:45.914284 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.911214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:49877 (local address 127.7.105.67:42963)
0809 00:37:45.911291 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.911294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.911409 (+ 115us) server_negotiation.cc:407] Connection header received
0809 00:37:45.911522 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.911524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.911552 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.911616 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:45.911807 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.912406 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.913421 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.913895 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.913963 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.913971 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.914156 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.914204 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:37:45.914369 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.911178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:34987 (local address 127.7.105.66:43307)
0809 00:37:45.911230 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.911233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.911373 (+ 140us) server_negotiation.cc:407] Connection header received
0809 00:37:45.911482 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.911484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.911494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.911553 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:45.911768 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.912322 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.913568 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.913959 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.914033 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.914042 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.914251 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.914297 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:45.914582 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.911078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:43307 (local address 127.7.105.65:34987)
0809 00:37:45.911275 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.911290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:45.911368 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:45.911588 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:45.911590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:45.911593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:45.911683 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.911686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.912421 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.912423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.913465 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:45.913468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.914057 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.914059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:45.914158 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.914163 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:45.914226 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:37:45.914270 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":39,"thread_start_us":73,"threads_started":1}
I20260809 00:37:45.914719 12630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f57a1016804e8483ef25fba95b6de2" candidate_uuid: "e363ae19997a472b9097f1f3c9409012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b706114ceb18421693db4baecd37ff83" is_pre_election: true
I20260809 00:37:45.914942 12630 raft_consensus.cc:2454] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e363ae19997a472b9097f1f3c9409012 in term 0.
I20260809 00:37:45.915076 12483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f57a1016804e8483ef25fba95b6de2" candidate_uuid: "e363ae19997a472b9097f1f3c9409012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71f02621074bcaa130e949835861fd" is_pre_election: true
I20260809 00:37:45.915230 12483 raft_consensus.cc:2454] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e363ae19997a472b9097f1f3c9409012 in term 0.
I20260809 00:37:45.915227 12256 leader_election.cc:302] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b706114ceb18421693db4baecd37ff83, e363ae19997a472b9097f1f3c9409012; no voters:
I20260809 00:37:45.915361 12711 raft_consensus.cc:2781] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:45.915445 12711 raft_consensus.cc:482] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:45.915472 12711 raft_consensus.cc:3037] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:45.916191 12711 raft_consensus.cc:504] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.916347 12711 leader_election.cc:288] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [CANDIDATE]: Term 1 election: Requested vote from peers b706114ceb18421693db4baecd37ff83 (127.7.105.67:42963), fc71f02621074bcaa130e949835861fd (127.7.105.66:43307)
I20260809 00:37:45.916523 12630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f57a1016804e8483ef25fba95b6de2" candidate_uuid: "e363ae19997a472b9097f1f3c9409012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b706114ceb18421693db4baecd37ff83"
I20260809 00:37:45.916558 12483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f57a1016804e8483ef25fba95b6de2" candidate_uuid: "e363ae19997a472b9097f1f3c9409012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71f02621074bcaa130e949835861fd"
I20260809 00:37:45.916635 12630 raft_consensus.cc:3037] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:45.916631 12483 raft_consensus.cc:3037] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:45.917413 12483 raft_consensus.cc:2454] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e363ae19997a472b9097f1f3c9409012 in term 1.
I20260809 00:37:45.917418 12630 raft_consensus.cc:2454] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e363ae19997a472b9097f1f3c9409012 in term 1.
I20260809 00:37:45.917582 12256 leader_election.cc:302] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b706114ceb18421693db4baecd37ff83, e363ae19997a472b9097f1f3c9409012; no voters:
I20260809 00:37:45.917718 12711 raft_consensus.cc:2781] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:45.917891 12711 raft_consensus.cc:686] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [term 1 LEADER]: Becoming Leader. State: Replica: e363ae19997a472b9097f1f3c9409012, State: Running, Role: LEADER
I20260809 00:37:45.918000 12711 consensus_queue.cc:237] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } }
I20260809 00:37:45.918959 12167 catalog_manager.cc:5495] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 reported cstate change: term changed from 0 to 1, leader changed from <none> to e363ae19997a472b9097f1f3c9409012 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "e363ae19997a472b9097f1f3c9409012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e363ae19997a472b9097f1f3c9409012" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41147 } health_report { overall_health: HEALTHY } } }
W20260809 00:37:45.919382 12684 tablet.cc:2369] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:45.929862 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.927264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56532 (local address 127.7.105.65:41147)
0809 00:37:45.927327 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:45.927329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:45.927354 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:37:45.927433 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:45.927433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:45.927454 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:45.927540 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:45.927689 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.928236 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.929010 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:45.929224 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:45.929300 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:45.929477 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:45.929599 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:45.929715 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:45.929755 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:37:45.929778 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:45.931901 12630 raft_consensus.cc:1261] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 1 FOLLOWER]: Refusing update from remote peer e363ae19997a472b9097f1f3c9409012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:45.931901 12483 raft_consensus.cc:1261] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 1 FOLLOWER]: Refusing update from remote peer e363ae19997a472b9097f1f3c9409012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:45.932503 12716 consensus_queue.cc:1035] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71f02621074bcaa130e949835861fd" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:45.932649 12711 consensus_queue.cc:1035] T 15f57a1016804e8483ef25fba95b6de2 P e363ae19997a472b9097f1f3c9409012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b706114ceb18421693db4baecd37ff83" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:45.935156 12720 mvcc.cc:205] Tried to move back new op lower bound from 7316422106853580800 to 7316422106800951296. Current Snapshot: MvccSnapshot[applied={T|T < 7316422106853580800}]
I20260809 00:37:45.935753 12721 mvcc.cc:205] Tried to move back new op lower bound from 7316422106853580800 to 7316422106800951296. Current Snapshot: MvccSnapshot[applied={T|T < 7316422106853580800}]
I20260809 00:37:45.937644 12168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:45.937742 12168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:45.938531 12168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f057d6bf1c44d06a39bde935d234542: Sending DeleteTablet for 3 replicas of tablet 15f57a1016804e8483ef25fba95b6de2
I20260809 00:37:45.938956 12605 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f57a1016804e8483ef25fba95b6de2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:45 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:39360
I20260809 00:37:45.938891 12461 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f57a1016804e8483ef25fba95b6de2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:45 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:52896
I20260809 00:37:45.939056 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12241
I20260809 00:37:45.941618 12730 tablet_replica.cc:330] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: stopping tablet replica
I20260809 00:37:45.941751 12730 raft_consensus.cc:2229] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:45.941896 12730 raft_consensus.cc:2258] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:45.942288 12729 tablet_replica.cc:330] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: stopping tablet replica
I20260809 00:37:45.942442 12729 raft_consensus.cc:2229] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:45.942560 12729 raft_consensus.cc:2258] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:45.942575 12730 ts_tablet_manager.cc:1905] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:45.943006 12729 ts_tablet_manager.cc:1905] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:45.944552 12730 ts_tablet_manager.cc:1918] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:45.944639 12730 log.cc:1196] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/15f57a1016804e8483ef25fba95b6de2
I20260809 00:37:45.944824 12729 ts_tablet_manager.cc:1918] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:45.944921 12729 log.cc:1196] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/15f57a1016804e8483ef25fba95b6de2
I20260809 00:37:45.944976 12730 ts_tablet_manager.cc:1939] T 15f57a1016804e8483ef25fba95b6de2 P b706114ceb18421693db4baecd37ff83: Deleting consensus metadata
I20260809 00:37:45.945253 12729 ts_tablet_manager.cc:1939] T 15f57a1016804e8483ef25fba95b6de2 P fc71f02621074bcaa130e949835861fd: Deleting consensus metadata
I20260809 00:37:45.945463 12147 catalog_manager.cc:4841] TS b706114ceb18421693db4baecd37ff83 (127.7.105.67:42963): tablet 15f57a1016804e8483ef25fba95b6de2 (table test-table [id=3faf019e58034cfbb809c65d58e7b9d3]) successfully deleted
I20260809 00:37:45.945674 12152 catalog_manager.cc:4841] TS fc71f02621074bcaa130e949835861fd (127.7.105.66:43307): tablet 15f57a1016804e8483ef25fba95b6de2 (table test-table [id=3faf019e58034cfbb809c65d58e7b9d3]) successfully deleted
W20260809 00:37:45.948295 12147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:41147 (error 108)
I20260809 00:37:45.948510 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12390
I20260809 00:37:45.953824 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:45.953671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41147 (local address 127.0.0.1:56544)
0809 00:37:45.953742 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:45.953792 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 00:37:45.954010 12147 catalog_manager.cc:4584] TS e363ae19997a472b9097f1f3c9409012 (127.7.105.65:41147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15f57a1016804e8483ef25fba95b6de2: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41147: connect: Connection refused (error 111)
I20260809 00:37:45.957908 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12541
I20260809 00:37:45.964385 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12130
2026-08-09T00:37:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2233 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T00:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:45Z Disabled control of system clock
I20260809 00:37:45.995668 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44889
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:37:46.079051 12735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:46.079272 12735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:46.079295 12735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:46.080595 12735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:46.080642 12735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:46.080655 12735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:46.080668 12735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:46.080681 12735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:46.082123 12735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:46.082404 12735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:46.082721 12735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:37:46.087965 12735 server_base.cc:1034] running on GCE node
W20260809 00:37:46.087940 12742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.087916 12744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.087940 12741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:46.088434 12735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:46.088799 12735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:46.089957 12735 hybrid_clock.cc:648] HybridClock initialized: now 1786235866089927 us; error 50 us; skew 500 ppm
I20260809 00:37:46.091248 12735 webserver.cc:460] Webserver started at http://127.7.105.126:41771/ using document root <none> and password file <none>
I20260809 00:37:46.091514 12735 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:46.091578 12735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:46.091744 12735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:46.092952 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f41cf4ee0b6f466ab274f850981f5fce"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "a0efb687a423786ee7675b946dd5cc0c"
server_key_iv: "53439977b4afe4a2ad8302aa343d017d"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.093341 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f41cf4ee0b6f466ab274f850981f5fce"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "a0efb687a423786ee7675b946dd5cc0c"
server_key_iv: "53439977b4afe4a2ad8302aa343d017d"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.094769 12735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:37:46.095685 12750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.095871 12735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:46.095978 12735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f41cf4ee0b6f466ab274f850981f5fce"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "a0efb687a423786ee7675b946dd5cc0c"
server_key_iv: "53439977b4afe4a2ad8302aa343d017d"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.096094 12735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:46.099488 12735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:46.099800 12735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:46.099954 12735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:46.104291 12735 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44889
I20260809 00:37:46.104323 12810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44889 every 8 connection(s)
I20260809 00:37:46.104681 12735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:46.105392 12811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:46.108217 12811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: Bootstrap starting.
I20260809 00:37:46.108776 12811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:46.109045 12811 log.cc:824] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:46.109687 12811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: No bootstrap required, opened a new log
I20260809 00:37:46.110284 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12735
I20260809 00:37:46.110387 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:46.110580 7589 external_mini_cluster.cc:1428] Setting key 8ac59cad8e095244cd4d71be47ffe626
I20260809 00:37:46.111893 12811 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } }
I20260809 00:37:46.112102 12811 raft_consensus.cc:374] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:46.112145 12811 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41cf4ee0b6f466ab274f850981f5fce, State: Initialized, Role: FOLLOWER
I20260809 00:37:46.112293 12811 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } }
I20260809 00:37:46.112390 12811 raft_consensus.cc:388] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:46.112423 12811 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:46.112473 12811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:46.113292 12811 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } }
I20260809 00:37:46.113394 12811 leader_election.cc:302] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f41cf4ee0b6f466ab274f850981f5fce; no voters:
I20260809 00:37:46.113476 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.111147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.7.105.126:44889)
0809 00:37:46.111359 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.111367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.111393 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:37:46.111869 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.111875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.111916 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.112035 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.112925 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.112929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.112929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.112933 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.112971 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.112983 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.113040 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.113131 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.113202 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260809 00:37:46.113583 12811 leader_election.cc:288] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:46.113867 12811 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:46.113929 12817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:46.114056 12817 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [term 1 LEADER]: Becoming Leader. State: Replica: f41cf4ee0b6f466ab274f850981f5fce, State: Running, Role: LEADER
I20260809 00:37:46.114151 12817 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } }
I20260809 00:37:46.114537 12818 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f41cf4ee0b6f466ab274f850981f5fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } } }
I20260809 00:37:46.114634 12818 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:46.114818 12819 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41cf4ee0b6f466ab274f850981f5fce. Latest consensus state: current_term: 1 leader_uuid: "f41cf4ee0b6f466ab274f850981f5fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cf4ee0b6f466ab274f850981f5fce" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44889 } } }
I20260809 00:37:46.114871 12827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:46.114883 12819 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:46.115449 12827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:46.117040 12827 catalog_manager.cc:1347] Generated new cluster ID: a5c002813f49495fbe328fede82988a8
I20260809 00:37:46.117117 12827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:46.146656 12827 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:46.146873 12827 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:46.156798 12827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: Generated new TSK 0
I20260809 00:37:46.157049 12827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:46.168852 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--builtin_ntp_servers=127.7.105.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:37:46.271651 12839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:46.271857 12839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:46.271878 12839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:46.273036 12839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:46.273078 12839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:46.273104 12839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:46.274372 12839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:46.274611 12839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:46.274891 12839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:46.280084 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.280114 12849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.280117 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:46.280397 12839 server_base.cc:1034] running on GCE node
I20260809 00:37:46.280650 12839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:46.280903 12839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:46.282058 12839 hybrid_clock.cc:648] HybridClock initialized: now 1786235866282046 us; error 33 us; skew 500 ppm
I20260809 00:37:46.283388 12839 webserver.cc:460] Webserver started at http://127.7.105.65:41085/ using document root <none> and password file <none>
I20260809 00:37:46.283674 12839 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:46.283725 12839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:46.283905 12839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:46.285082 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "97700e15cb674ecbb1e2920b20c588dd"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "d605e6244357bb37c85c0eca033f48d0"
server_key_iv: "98ffab973b5c3e7622630da3e3a35a65"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.285459 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "97700e15cb674ecbb1e2920b20c588dd"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "d605e6244357bb37c85c0eca033f48d0"
server_key_iv: "98ffab973b5c3e7622630da3e3a35a65"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.286811 12839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:46.287644 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.287818 12839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:46.287925 12839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "97700e15cb674ecbb1e2920b20c588dd"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "d605e6244357bb37c85c0eca033f48d0"
server_key_iv: "98ffab973b5c3e7622630da3e3a35a65"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.288009 12839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:46.291327 12839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:46.291604 12839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:46.291749 12839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:46.291961 12839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:46.292326 12839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:46.292389 12839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.292429 12839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:46.292457 12839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.298229 12839 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:41475
I20260809 00:37:46.298303 12981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:41475 every 8 connection(s)
I20260809 00:37:46.298642 12839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:46.301657 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.299414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:41557 (local address 127.7.105.126:44889)
0809 00:37:46.299481 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.299483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.299711 (+ 228us) server_negotiation.cc:407] Connection header received
0809 00:37:46.300306 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.300307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.300315 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.300369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.301471 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.301473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.301473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.301476 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.301491 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.301495 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.301542 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.301589 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.301604 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:37:46.301844 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.299422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44889 (local address 127.7.105.65:41557)
0809 00:37:46.299592 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.299624 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.300216 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.300422 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.300425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.300434 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.301323 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.301324 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.301352 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.301380 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.301541 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.301543 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.301546 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.301597 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.301645 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260809 00:37:46.302517 12982 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44889
I20260809 00:37:46.302629 12982 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:46.302783 12982 heartbeater.cc:511] Master 127.7.105.126:44889 requested a full tablet report, sending...
I20260809 00:37:46.303401 12770 ts_manager.cc:194] Registered new tserver with Master: 97700e15cb674ecbb1e2920b20c588dd (127.7.105.65:41475)
I20260809 00:37:46.305294 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12839
I20260809 00:37:46.305379 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:46.305521 7589 external_mini_cluster.cc:1428] Setting key fc2fcc0e697d911de27624e0291562fa
I20260809 00:37:46.306376 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--builtin_ntp_servers=127.7.105.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:37:46.392421 12986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:46.392617 12986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:46.392640 12986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:46.393759 12986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:46.393795 12986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:46.393821 12986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:46.395102 12986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:46.395289 12986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:46.395534 12986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:46.400966 12991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.401073 12992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.400993 12994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:46.401144 12986 server_base.cc:1034] running on GCE node
I20260809 00:37:46.401404 12986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:46.401674 12986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:46.402843 12986 hybrid_clock.cc:648] HybridClock initialized: now 1786235866402836 us; error 35 us; skew 500 ppm
I20260809 00:37:46.404237 12986 webserver.cc:460] Webserver started at http://127.7.105.66:45301/ using document root <none> and password file <none>
I20260809 00:37:46.404505 12986 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:46.404577 12986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:46.404803 12986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:46.405903 12986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "e32278d064964f3087f963211dd1909f"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "bad03dc0cac1b6e43a2528c765285e59"
server_key_iv: "2428820de78a67d2d18ed9db9a12bbd8"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.406288 12986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "e32278d064964f3087f963211dd1909f"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "bad03dc0cac1b6e43a2528c765285e59"
server_key_iv: "2428820de78a67d2d18ed9db9a12bbd8"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.407629 12986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:46.408571 13000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.408812 12986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:46.408914 12986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "e32278d064964f3087f963211dd1909f"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "bad03dc0cac1b6e43a2528c765285e59"
server_key_iv: "2428820de78a67d2d18ed9db9a12bbd8"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.408985 12986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:46.412536 12986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:46.412799 12986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:46.412945 12986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:46.413166 12986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:46.413527 12986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:46.413581 12986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.413621 12986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:46.413650 12986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.419749 12986 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:37207
I20260809 00:37:46.419796 13121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:37207 every 8 connection(s)
I20260809 00:37:46.420238 12986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:46.422083 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 12986
I20260809 00:37:46.422196 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:46.422381 7589 external_mini_cluster.cc:1428] Setting key 90fa17eae0eb9cce100f02ed4f027473
I20260809 00:37:46.423457 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--builtin_ntp_servers=127.7.105.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 00:37:46.423555 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.421003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37391 (local address 127.7.105.126:44889)
0809 00:37:46.421085 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.421087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.421327 (+ 240us) server_negotiation.cc:407] Connection header received
0809 00:37:46.421785 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.421786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.421791 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.421902 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.423236 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.423238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.423238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.423242 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.423263 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.423268 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.423333 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.423401 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.423417 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:37:46.423719 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.420990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44889 (local address 127.7.105.66:37391)
0809 00:37:46.421196 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.421238 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.421699 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.421955 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.421958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.421967 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.423064 (+ 1097us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.423065 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.423106 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.423145 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.423340 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.423346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.423348 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.423408 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.423461 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260809 00:37:46.424199 13122 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44889
I20260809 00:37:46.424306 13122 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:46.424476 13122 heartbeater.cc:511] Master 127.7.105.126:44889 requested a full tablet report, sending...
I20260809 00:37:46.424860 12770 ts_manager.cc:194] Registered new tserver with Master: e32278d064964f3087f963211dd1909f (127.7.105.66:37207)
W20260809 00:37:46.507135 13125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:46.507385 13125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:46.507426 13125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:46.509029 13125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:46.509106 13125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:46.509150 13125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:46.510691 13125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:46.510947 13125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:46.511276 13125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:46.517285 13133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:46.517323 13132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:46.517396 13125 server_base.cc:1034] running on GCE node
W20260809 00:37:46.517303 13135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:46.517763 13125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:46.517966 13125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:46.519104 13125 hybrid_clock.cc:648] HybridClock initialized: now 1786235866519087 us; error 27 us; skew 500 ppm
I20260809 00:37:46.520447 13125 webserver.cc:460] Webserver started at http://127.7.105.67:45607/ using document root <none> and password file <none>
I20260809 00:37:46.520694 13125 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:46.520744 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:46.520882 13125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:46.521971 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "a4b4de9d1af446628a6468d6973d4913"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "0b743dfcfc8c7667eaf5d3e1289221a2"
server_key_iv: "e6430ec6088c9ffbfae85515fa25f9db"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.522346 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "a4b4de9d1af446628a6468d6973d4913"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "0b743dfcfc8c7667eaf5d3e1289221a2"
server_key_iv: "e6430ec6088c9ffbfae85515fa25f9db"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.523670 13125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:46.524492 13141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.524688 13125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:46.524796 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "a4b4de9d1af446628a6468d6973d4913"
format_stamp: "Formatted at 2026-08-09 00:37:46 on dist-test-slave-1zqn"
server_key: "0b743dfcfc8c7667eaf5d3e1289221a2"
server_key_iv: "e6430ec6088c9ffbfae85515fa25f9db"
server_key_version: "encryptionkey@0"
I20260809 00:37:46.524870 13125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:46.528272 13125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:46.528523 13125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:46.528685 13125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:46.528898 13125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:46.529214 13125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:46.529269 13125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.529309 13125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:46.529335 13125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:46.535025 13125 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:33599
I20260809 00:37:46.535069 13267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:33599 every 8 connection(s)
I20260809 00:37:46.535388 13125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:46.538321 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.536116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:39807 (local address 127.7.105.126:44889)
0809 00:37:46.536171 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.536173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.536425 (+ 252us) server_negotiation.cc:407] Connection header received
0809 00:37:46.536863 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.536864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.536870 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.536983 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.538083 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.538085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.538085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.538089 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.538105 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.538110 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.538171 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.538254 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.538268 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:46.538493 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.536113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44889 (local address 127.7.105.67:39807)
0809 00:37:46.536306 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.536339 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.536781 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.537030 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.537033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.537041 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.537935 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.537935 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.537963 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.537992 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.538177 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.538181 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.538184 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.538243 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.538292 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 00:37:46.538995 13268 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44889
I20260809 00:37:46.539091 13268 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:46.539208 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13125
I20260809 00:37:46.539240 13268 heartbeater.cc:511] Master 127.7.105.126:44889 requested a full tablet report, sending...
I20260809 00:37:46.539304 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:46.539485 7589 external_mini_cluster.cc:1428] Setting key 215e17d6d6a65c4dc0dff9cb02b80b88
I20260809 00:37:46.539660 12770 ts_manager.cc:194] Registered new tserver with Master: a4b4de9d1af446628a6468d6973d4913 (127.7.105.67:33599)
I20260809 00:37:46.540366 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:37:46.542647 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.541717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.7.105.126:44889)
0809 00:37:46.541772 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.541774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.541877 (+ 103us) server_negotiation.cc:407] Connection header received
0809 00:37:46.541933 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.541933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.541938 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.541990 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.542484 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.542486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.542486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.542489 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.542501 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.542504 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.542554 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.542590 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.542601 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:37:46.544276 12769 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:46.545853 12769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:46.550071 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.7.105.65:41475)
0809 00:37:46.548553 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.548558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.548571 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:37:46.548604 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.548606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.548616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.548713 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.549774 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.549783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.549783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.549787 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.549818 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.549845 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.549908 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.549967 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.549981 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260809 00:37:46.550083 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41475 (local address 127.0.0.1:33620)
0809 00:37:46.548402 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.548461 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.548522 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.548726 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.548732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.548739 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.549244 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.549245 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.549263 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.549289 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.549904 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.549905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.549906 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.549961 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.549976 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":76,"threads_started":1}
I20260809 00:37:46.550087 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:37207 (local address 127.0.0.1:47810)
0809 00:37:46.548651 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.548658 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.548711 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.549186 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.549187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.549190 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.549601 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.549602 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.549609 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.549620 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.549877 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.549885 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.549888 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.549956 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.549971 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":68,"threads_started":1}
I20260809 00:37:46.550330 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.7.105.66:37207)
0809 00:37:46.548628 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.548632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.548971 (+ 339us) server_negotiation.cc:407] Connection header received
0809 00:37:46.549008 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.549010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.549016 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.549104 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.549707 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.549715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.549716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.549720 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.549764 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.549790 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.549849 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.550207 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.550226 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":87,"threads_started":1}
I20260809 00:37:46.550678 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:33599 (local address 127.0.0.1:49668)
0809 00:37:46.548530 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.548539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.548575 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.549451 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.549453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.549455 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.549764 (+ 309us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.549765 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.549770 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.549780 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.550027 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.550028 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.550029 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.550058 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.550068 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":110,"threads_started":1}
I20260809 00:37:46.550951 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.548181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.7.105.67:33599)
0809 00:37:46.548827 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.548832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.548842 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:46.548867 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.548869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.548879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.549428 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.549863 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.549869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.549870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.549874 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.549925 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.549947 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.550123 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.550146 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.550853 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":87,"threads_started":1}
I20260809 00:37:46.550907 12907 tablet_service.cc:1467] Processing CreateTablet for tablet 36caa960343145e79b5de199bbc2e957 (DEFAULT_TABLE table=test-table [id=ad84d50c571847b0bc3d09282585a236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:46.551097 13050 tablet_service.cc:1467] Processing CreateTablet for tablet 36caa960343145e79b5de199bbc2e957 (DEFAULT_TABLE table=test-table [id=ad84d50c571847b0bc3d09282585a236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:46.551329 12907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36caa960343145e79b5de199bbc2e957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:46.551470 13050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36caa960343145e79b5de199bbc2e957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:46.551542 13197 tablet_service.cc:1467] Processing CreateTablet for tablet 36caa960343145e79b5de199bbc2e957 (DEFAULT_TABLE table=test-table [id=ad84d50c571847b0bc3d09282585a236]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:46.551863 13197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36caa960343145e79b5de199bbc2e957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:46.554116 13284 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Bootstrap starting.
I20260809 00:37:46.554842 13284 tablet_bootstrap.cc:654] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:46.554926 13285 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Bootstrap starting.
I20260809 00:37:46.555469 13284 log.cc:824] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:46.555586 13285 tablet_bootstrap.cc:654] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:46.555924 13285 log.cc:824] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:46.556305 13284 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: No bootstrap required, opened a new log
I20260809 00:37:46.556391 13284 ts_tablet_manager.cc:1397] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:46.556639 13285 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: No bootstrap required, opened a new log
I20260809 00:37:46.556694 13285 ts_tablet_manager.cc:1397] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:46.558506 13284 raft_consensus.cc:348] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.558673 13284 raft_consensus.cc:374] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:46.558723 13284 raft_consensus.cc:729] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97700e15cb674ecbb1e2920b20c588dd, State: Initialized, Role: FOLLOWER
I20260809 00:37:46.558676 13285 raft_consensus.cc:348] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.558863 13285 raft_consensus.cc:374] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:46.558908 13285 raft_consensus.cc:729] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32278d064964f3087f963211dd1909f, State: Initialized, Role: FOLLOWER
I20260809 00:37:46.558881 13284 consensus_queue.cc:260] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.559047 13285 consensus_queue.cc:260] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.559193 13284 ts_tablet_manager.cc:1428] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:46.559231 12982 heartbeater.cc:503] Master 127.7.105.126:44889 was elected leader, sending a full tablet report...
I20260809 00:37:46.555868 13286 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Bootstrap starting.
I20260809 00:37:46.559345 13285 ts_tablet_manager.cc:1428] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:46.559388 13122 heartbeater.cc:503] Master 127.7.105.126:44889 was elected leader, sending a full tablet report...
I20260809 00:37:46.560004 13286 tablet_bootstrap.cc:654] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:46.560345 13286 log.cc:824] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:46.561074 13286 tablet_bootstrap.cc:492] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: No bootstrap required, opened a new log
I20260809 00:37:46.561131 13286 ts_tablet_manager.cc:1397] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 00:37:46.562872 13286 raft_consensus.cc:348] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.562996 13286 raft_consensus.cc:374] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:46.563019 13286 raft_consensus.cc:729] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4b4de9d1af446628a6468d6973d4913, State: Initialized, Role: FOLLOWER
I20260809 00:37:46.563139 13286 consensus_queue.cc:260] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.563369 13286 ts_tablet_manager.cc:1428] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:37:46.563444 13268 heartbeater.cc:503] Master 127.7.105.126:44889 was elected leader, sending a full tablet report...
W20260809 00:37:46.670842 13123 tablet.cc:2369] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:37:46.785938 13270 tablet.cc:2369] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:37:46.799325 12983 tablet.cc:2369] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:46.832259 13295 raft_consensus.cc:482] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:46.832401 13295 raft_consensus.cc:504] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.832737 13295 leader_election.cc:288] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97700e15cb674ecbb1e2920b20c588dd (127.7.105.65:41475), e32278d064964f3087f963211dd1909f (127.7.105.66:37207)
I20260809 00:37:46.834285 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.832972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:37207 (local address 127.7.105.67:50641)
0809 00:37:46.833041 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.833079 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.833124 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.833300 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.833301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.833304 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.833816 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.833817 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.833826 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.833840 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.834110 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.834111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.834112 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.834170 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.834185 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 00:37:46.834309 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.833055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:50641 (local address 127.7.105.66:37207)
0809 00:37:46.833127 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.833129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.833180 (+ 51us) server_negotiation.cc:407] Connection header received
0809 00:37:46.833212 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.833213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.833220 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.833276 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.833944 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.833958 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.833959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.833964 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.833986 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.833991 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.834083 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.834209 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.834226 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:37:46.834375 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.833055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:58085 (local address 127.7.105.65:41475)
0809 00:37:46.833107 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.833109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.833285 (+ 176us) server_negotiation.cc:407] Connection header received
0809 00:37:46.833319 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.833321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.833328 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.833392 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.834124 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.834125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.834126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.834130 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.834147 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.834152 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.834209 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.834285 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.834299 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:46.834554 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.832972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41475 (local address 127.7.105.67:58085)
0809 00:37:46.833185 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.833198 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:46.833258 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:46.833403 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:46.833404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:46.833405 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:46.834003 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:37:46.834003 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:46.834014 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:37:46.834026 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:46.834228 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:46.834228 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:46.834229 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:46.834278 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:37:46.834290 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":22,"thread_start_us":103,"threads_started":1}
I20260809 00:37:46.834906 12929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36caa960343145e79b5de199bbc2e957" candidate_uuid: "a4b4de9d1af446628a6468d6973d4913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97700e15cb674ecbb1e2920b20c588dd" is_pre_election: true
I20260809 00:37:46.834898 13074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36caa960343145e79b5de199bbc2e957" candidate_uuid: "a4b4de9d1af446628a6468d6973d4913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32278d064964f3087f963211dd1909f" is_pre_election: true
I20260809 00:37:46.835070 13074 raft_consensus.cc:2454] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4b4de9d1af446628a6468d6973d4913 in term 0.
I20260809 00:37:46.835067 12929 raft_consensus.cc:2454] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4b4de9d1af446628a6468d6973d4913 in term 0.
I20260809 00:37:46.835305 13146 leader_election.cc:302] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97700e15cb674ecbb1e2920b20c588dd, a4b4de9d1af446628a6468d6973d4913; no voters:
I20260809 00:37:46.835453 13295 raft_consensus.cc:2781] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:46.835553 13295 raft_consensus.cc:482] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:46.835592 13295 raft_consensus.cc:3037] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:46.836421 13295 raft_consensus.cc:504] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.836578 13295 leader_election.cc:288] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [CANDIDATE]: Term 1 election: Requested vote from peers 97700e15cb674ecbb1e2920b20c588dd (127.7.105.65:41475), e32278d064964f3087f963211dd1909f (127.7.105.66:37207)
I20260809 00:37:46.836756 12929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36caa960343145e79b5de199bbc2e957" candidate_uuid: "a4b4de9d1af446628a6468d6973d4913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97700e15cb674ecbb1e2920b20c588dd"
I20260809 00:37:46.836756 13074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36caa960343145e79b5de199bbc2e957" candidate_uuid: "a4b4de9d1af446628a6468d6973d4913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32278d064964f3087f963211dd1909f"
I20260809 00:37:46.836843 13074 raft_consensus.cc:3037] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:46.836843 12929 raft_consensus.cc:3037] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:46.837782 12929 raft_consensus.cc:2454] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4b4de9d1af446628a6468d6973d4913 in term 1.
I20260809 00:37:46.837782 13074 raft_consensus.cc:2454] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4b4de9d1af446628a6468d6973d4913 in term 1.
I20260809 00:37:46.837993 13145 leader_election.cc:302] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4b4de9d1af446628a6468d6973d4913, e32278d064964f3087f963211dd1909f; no voters:
I20260809 00:37:46.838151 13295 raft_consensus.cc:2781] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:46.838331 13295 raft_consensus.cc:686] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 1 LEADER]: Becoming Leader. State: Replica: a4b4de9d1af446628a6468d6973d4913, State: Running, Role: LEADER
I20260809 00:37:46.838462 13295 consensus_queue.cc:237] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } }
I20260809 00:37:46.839361 12769 catalog_manager.cc:5495] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4b4de9d1af446628a6468d6973d4913 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "a4b4de9d1af446628a6468d6973d4913" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4b4de9d1af446628a6468d6973d4913" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:46.908129 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.906962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.7.105.67:33599)
0809 00:37:46.907027 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:46.907029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:46.907038 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:46.907121 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:46.907122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:46.907143 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:46.907210 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:46.907872 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:46.907874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:46.907874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:46.907879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:46.907895 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:46.907900 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:46.907952 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:46.908000 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:37:46.908015 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:37:46.910080 12929 raft_consensus.cc:1261] T 36caa960343145e79b5de199bbc2e957 P 97700e15cb674ecbb1e2920b20c588dd [term 1 FOLLOWER]: Refusing update from remote peer a4b4de9d1af446628a6468d6973d4913: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:46.910084 13074 raft_consensus.cc:1261] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 1 FOLLOWER]: Refusing update from remote peer a4b4de9d1af446628a6468d6973d4913: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:46.910467 13297 consensus_queue.cc:1035] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e32278d064964f3087f963211dd1909f" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 37207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:46.910652 13295 consensus_queue.cc:1035] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97700e15cb674ecbb1e2920b20c588dd" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:46.913278 13301 mvcc.cc:205] Tried to move back new op lower bound from 7316422110860783616 to 7316422110571032576. Current Snapshot: MvccSnapshot[applied={T|T < 7316422110860783616}]
I20260809 00:37:46.915830 12768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:46.915944 12768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:46.916848 12768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f41cf4ee0b6f466ab274f850981f5fce: Sending DeleteTablet for 3 replicas of tablet 36caa960343145e79b5de199bbc2e957
I20260809 00:37:46.917307 13050 tablet_service.cc:1514] Processing DeleteTablet for tablet 36caa960343145e79b5de199bbc2e957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:46 UTC) from {username='slave'} at 127.0.0.1:47810
I20260809 00:37:46.917289 13197 tablet_service.cc:1514] Processing DeleteTablet for tablet 36caa960343145e79b5de199bbc2e957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:46 UTC) from {username='slave'} at 127.0.0.1:49668
I20260809 00:37:46.917315 12907 tablet_service.cc:1514] Processing DeleteTablet for tablet 36caa960343145e79b5de199bbc2e957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:46 UTC) from {username='slave'} at 127.0.0.1:33620
I20260809 00:37:46.917594 13315 tablet_replica.cc:330] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: stopping tablet replica
I20260809 00:37:46.917695 13315 raft_consensus.cc:2229] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:46.917804 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12839
I20260809 00:37:46.929344 13315 raft_consensus.cc:2258] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:46.929519 13313 tablet_replica.cc:330] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: stopping tablet replica
I20260809 00:37:46.929656 13313 raft_consensus.cc:2229] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:46.929776 13313 raft_consensus.cc:2258] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:46.931118 13315 ts_tablet_manager.cc:1905] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:46.932778 13315 ts_tablet_manager.cc:1918] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:46.932794 13313 ts_tablet_manager.cc:1905] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:46.932960 13315 log.cc:1196] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/36caa960343145e79b5de199bbc2e957
I20260809 00:37:46.933338 13315 ts_tablet_manager.cc:1939] T 36caa960343145e79b5de199bbc2e957 P e32278d064964f3087f963211dd1909f: Deleting consensus metadata
I20260809 00:37:46.934070 12753 catalog_manager.cc:4841] TS e32278d064964f3087f963211dd1909f (127.7.105.66:37207): tablet 36caa960343145e79b5de199bbc2e957 (table test-table [id=ad84d50c571847b0bc3d09282585a236]) successfully deleted
I20260809 00:37:46.934520 13313 ts_tablet_manager.cc:1918] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:46.934624 13313 log.cc:1196] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/36caa960343145e79b5de199bbc2e957
I20260809 00:37:46.934876 13313 ts_tablet_manager.cc:1939] T 36caa960343145e79b5de199bbc2e957 P a4b4de9d1af446628a6468d6973d4913: Deleting consensus metadata
I20260809 00:37:46.935305 12752 catalog_manager.cc:4841] TS a4b4de9d1af446628a6468d6973d4913 (127.7.105.67:33599): tablet 36caa960343145e79b5de199bbc2e957 (table test-table [id=ad84d50c571847b0bc3d09282585a236]) successfully deleted
W20260809 00:37:46.936152 12754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:41475 (error 108)
I20260809 00:37:46.936422 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12986
I20260809 00:37:46.937878 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:46.936561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41475 (local address 127.0.0.1:33632)
0809 00:37:46.937794 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:46.937845 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1207}
W20260809 00:37:46.938369 12754 catalog_manager.cc:4584] TS 97700e15cb674ecbb1e2920b20c588dd (127.7.105.65:41475): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36caa960343145e79b5de199bbc2e957: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41475: connect: Connection refused (error 111)
I20260809 00:37:46.952889 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13125
I20260809 00:37:46.959358 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 12735
2026-08-09T00:37:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (992 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 00:37:46.980557 7589 mini_oidc.cc:152] Starting JWKS server
I20260809 00:37:46.981769 7589 webserver.cc:460] Webserver started at https://127.0.0.1:38297/ using document root <none> and password file <none>
I20260809 00:37:46.981897 7589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:37:46.982089 7589 webserver.cc:460] Webserver started at http://127.0.0.1:35287/ using document root <none> and password file <none>
2026-08-09T00:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:46Z Disabled control of system clock
I20260809 00:37:46.991497 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45363
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:36213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45363
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38297/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/testchainca.cert with env {}
W20260809 00:37:47.074127 13324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:47.074345 13324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:47.074368 13324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:47.075583 13324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:47.075624 13324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:47.075636 13324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:47.075649 13324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:47.075662 13324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:37:47.075675 13324 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:37:47.075685 13324 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38297/jwks.json
I20260809 00:37:47.077265 13324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38297/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:47.077482 13324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:47.077658 13324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:47.082902 13332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:47.082994 13324 server_base.cc:1034] running on GCE node
W20260809 00:37:47.083010 13335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.082908 13331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:47.083424 13324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:47.083734 13324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:47.084880 13324 hybrid_clock.cc:648] HybridClock initialized: now 1786235867084861 us; error 39 us; skew 500 ppm
I20260809 00:37:47.086299 13324 webserver.cc:460] Webserver started at http://127.7.105.126:33613/ using document root <none> and password file <none>
I20260809 00:37:47.086581 13324 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:47.086633 13324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:47.086783 13324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:47.087903 13324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "a42843783864459bbf395199c320f588"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "14bd28e3419ee5fbf2291b42703add08"
server_key_iv: "116f9af0666c5832a70adf9aff658b61"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.088354 13324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "a42843783864459bbf395199c320f588"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "14bd28e3419ee5fbf2291b42703add08"
server_key_iv: "116f9af0666c5832a70adf9aff658b61"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.089745 13324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:47.090533 13341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:47.090711 13324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:47.090816 13324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a42843783864459bbf395199c320f588"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "14bd28e3419ee5fbf2291b42703add08"
server_key_iv: "116f9af0666c5832a70adf9aff658b61"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.090898 13324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:47.114179 13324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:47.114593 13324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:47.114770 13324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:47.119333 13324 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:45363
I20260809 00:37:47.119369 13400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:45363 every 8 connection(s)
I20260809 00:37:47.119748 13324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:47.120836 13401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:47.123127 13401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588: Bootstrap starting.
I20260809 00:37:47.123715 13401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:47.123963 13401 log.cc:824] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:47.124687 13401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588: No bootstrap required, opened a new log
I20260809 00:37:47.126276 13401 raft_consensus.cc:348] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } }
I20260809 00:37:47.126436 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13324
I20260809 00:37:47.126477 13401 raft_consensus.cc:374] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:47.126519 13401 raft_consensus.cc:729] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42843783864459bbf395199c320f588, State: Initialized, Role: FOLLOWER
I20260809 00:37:47.126542 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:47.126680 13401 consensus_queue.cc:260] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } }
I20260809 00:37:47.126726 7589 external_mini_cluster.cc:1428] Setting key 3e9702c96bb4cfd1d80331685a10f722
I20260809 00:37:47.126767 13401 raft_consensus.cc:388] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:47.126804 13401 raft_consensus.cc:482] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:47.126856 13401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:47.127621 13401 raft_consensus.cc:504] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } }
I20260809 00:37:47.127715 13401 leader_election.cc:302] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a42843783864459bbf395199c320f588; no voters:
I20260809 00:37:47.127905 13401 leader_election.cc:288] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:47.127938 13406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:47.128208 13406 raft_consensus.cc:686] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [term 1 LEADER]: Becoming Leader. State: Replica: a42843783864459bbf395199c320f588, State: Running, Role: LEADER
I20260809 00:37:47.128216 13401 sys_catalog.cc:564] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:47.128404 13406 consensus_queue.cc:237] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } }
I20260809 00:37:47.128847 13407 sys_catalog.cc:455] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a42843783864459bbf395199c320f588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } } }
I20260809 00:37:47.128918 13407 sys_catalog.cc:458] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.128824 13406 sys_catalog.cc:455] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a42843783864459bbf395199c320f588. Latest consensus state: current_term: 1 leader_uuid: "a42843783864459bbf395199c320f588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42843783864459bbf395199c320f588" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45363 } } }
I20260809 00:37:47.129019 13406 sys_catalog.cc:458] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.129850 13416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:47.130554 13416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:47.132323 13416 catalog_manager.cc:1347] Generated new cluster ID: 81620ea2cce34ec0a7e7038da0c24a62
I20260809 00:37:47.132375 13416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:47.133620 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.127244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45776 (local address 127.7.105.126:45363)
0809 00:37:47.127467 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.127474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.127500 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:37:47.127607 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.127613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.127702 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.127866 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:47.128142 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.130027 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.131179 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.131740 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.131837 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.132854 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:47.132858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:47.132858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:47.133081 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:47.133134 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:47.133147 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:47.133183 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:47.133312 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:37:47.133404 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260809 00:37:47.137116 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:47.144667 13416 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:47.145383 13416 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:47.153921 13416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a42843783864459bbf395199c320f588: Generated new TSK 0
I20260809 00:37:47.154152 13416 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:37:47.181967 13434 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 00:37:47.182137 13434 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 00:37:47.182762 13433 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:47.166239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45363 (local address 127.0.0.1:45792)
0809 00:37:47.166435 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:47.166446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:47.166507 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:47.166708 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:47.166709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:47.166721 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:37:47.166790 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.166792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.167728 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.167728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.168416 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.168420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.168891 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.168892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.168937 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.168958 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:37:47.182223 (+ 13265us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:37:47.182328 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.105.126:45363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":106,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.105.126:45363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 00:37:47.183107 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.166323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.7.105.126:45363)
0809 00:37:47.166379 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.166381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.166514 (+ 133us) server_negotiation.cc:407] Connection header received
0809 00:37:47.166607 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.166608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.166623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.166695 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:37:47.166866 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.167643 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.168501 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.168809 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.168885 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.169070 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:37:47.182059 (+ 12989us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:37:47.183034 (+ 975us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 00:37:47.183154 13405 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
I20260809 00:37:47.183161 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13324
2026-08-09T00:37:47Z chronyd exiting
I20260809 00:37:47.222375 7589 test_util.cc:175] -----------------------------------------------
I20260809 00:37:47.222483 7589 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235843038573-7589-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 00:37:47.223279 7589 mini_oidc.cc:152] Starting JWKS server
I20260809 00:37:47.223985 7589 webserver.cc:460] Webserver started at https://127.0.0.1:38681/ using document root <none> and password file <none>
I20260809 00:37:47.224083 7589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:37:47.224288 7589 webserver.cc:460] Webserver started at http://127.0.0.1:44445/ using document root <none> and password file <none>
2026-08-09T00:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:47Z Disabled control of system clock
I20260809 00:37:47.233304 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36599
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36599
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38681/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/testchainca.cert with env {}
W20260809 00:37:47.315243 13443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:47.315467 13443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:47.315491 13443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:47.316664 13443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:47.316705 13443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:47.316718 13443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:47.316731 13443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:47.316743 13443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:37:47.316756 13443 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:37:47.316766 13443 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38681/jwks.json
I20260809 00:37:47.318286 13443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38681/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:47.318499 13443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:47.318688 13443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:47.323760 13450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.323777 13455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.323798 13449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:47.323983 13443 server_base.cc:1034] running on GCE node
I20260809 00:37:47.324250 13443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:47.324551 13443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:47.325716 13443 hybrid_clock.cc:648] HybridClock initialized: now 1786235867325700 us; error 45 us; skew 500 ppm
I20260809 00:37:47.327083 13443 webserver.cc:460] Webserver started at http://127.7.105.126:40883/ using document root <none> and password file <none>
I20260809 00:37:47.327346 13443 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:47.327419 13443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:47.327574 13443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:47.328806 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "049639c3c64945148f89881fd32cff3f"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "7529cec3d0942eecacbccc7fba23a63d"
server_key_iv: "edc091ec76ee9e46355413948bcdb39e"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.329211 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "049639c3c64945148f89881fd32cff3f"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "7529cec3d0942eecacbccc7fba23a63d"
server_key_iv: "edc091ec76ee9e46355413948bcdb39e"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.330549 13443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:37:47.331265 13461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:47.331434 13443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:47.331566 13443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "049639c3c64945148f89881fd32cff3f"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "7529cec3d0942eecacbccc7fba23a63d"
server_key_iv: "edc091ec76ee9e46355413948bcdb39e"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.331645 13443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:47.342171 13443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:47.342482 13443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:47.342639 13443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:47.346751 13443 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:36599
I20260809 00:37:47.346787 13518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:36599 every 8 connection(s)
I20260809 00:37:47.347177 13443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:47.347389 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13443
I20260809 00:37:47.347501 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:47.347692 7589 external_mini_cluster.cc:1428] Setting key 5f03e4e9fabe04c68696e65590098c17
I20260809 00:37:47.348426 13519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:47.351663 13519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f: Bootstrap starting.
I20260809 00:37:47.352439 13519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:47.352748 13519 log.cc:824] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:47.353595 13519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f: No bootstrap required, opened a new log
I20260809 00:37:47.353931 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.348272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.7.105.126:36599)
0809 00:37:47.348536 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.348546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.348595 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:37:47.349174 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.349185 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.349269 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.349432 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:47.349610 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.350942 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.351676 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.352307 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.352441 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.353152 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:47.353155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:47.353156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:47.353370 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:47.353415 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:47.353430 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:47.353501 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:47.353574 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 00:37:47.353720 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260809 00:37:47.355055 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:47.355819 13519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } }
I20260809 00:37:47.355955 13519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:47.356014 13519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049639c3c64945148f89881fd32cff3f, State: Initialized, Role: FOLLOWER
I20260809 00:37:47.356199 13519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } }
I20260809 00:37:47.356271 13519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:47.356292 13519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:47.356325 13519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:47.356850 13519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } }
I20260809 00:37:47.356933 13519 leader_election.cc:302] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049639c3c64945148f89881fd32cff3f; no voters:
I20260809 00:37:47.357069 13519 leader_election.cc:288] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:47.357151 13526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:47.357298 13526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [term 1 LEADER]: Becoming Leader. State: Replica: 049639c3c64945148f89881fd32cff3f, State: Running, Role: LEADER
I20260809 00:37:47.357360 13519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:47.357374 13526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } }
I20260809 00:37:47.357699 13528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049639c3c64945148f89881fd32cff3f. Latest consensus state: current_term: 1 leader_uuid: "049639c3c64945148f89881fd32cff3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } } }
I20260809 00:37:47.357712 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "049639c3c64945148f89881fd32cff3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049639c3c64945148f89881fd32cff3f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36599 } } }
I20260809 00:37:47.357872 13528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.357888 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.358332 13535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:47.358878 13535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:47.360489 13535 catalog_manager.cc:1347] Generated new cluster ID: 698fdb38e3af4759a610a9b556632d47
I20260809 00:37:47.360561 13535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:47.371790 13535 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:47.372573 13535 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:47.379474 13535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 049639c3c64945148f89881fd32cff3f: Generated new TSK 0
I20260809 00:37:47.379726 13535 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:37:47.414906 13554 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 00:37:47.415023 13554 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:37:47.415323 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.393055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.7.105.126:36599)
0809 00:37:47.393124 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.393126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.393219 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:37:47.393299 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.393300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.393311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.393373 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:37:47.393549 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.394074 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.395171 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.395354 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.395398 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.395585 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:37:47.415088 (+ 19503us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:37:47.415265 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 00:37:47.415400 13522 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 00:37:47.415443 13553 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:47.392985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:36599 (local address 127.0.0.1:45358)
0809 00:37:47.393146 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:47.393157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:47.393225 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:47.393411 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:47.393413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:47.393414 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:37:47.393476 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.393478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.394148 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.394149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.395082 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.395085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.395418 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.395419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.395482 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.395487 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:37:47.415223 (+ 19736us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:37:47.415342 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.105.126:36599: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260809 00:37:47.415798 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13443
2026-08-09T00:37:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 00:37:47.457470 7589 mini_oidc.cc:152] Starting JWKS server
I20260809 00:37:47.458223 7589 webserver.cc:460] Webserver started at https://127.0.0.1:36341/ using document root <none> and password file <none>
I20260809 00:37:47.458267 7589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:37:47.458400 7589 webserver.cc:460] Webserver started at http://127.0.0.1:38499/ using document root <none> and password file <none>
2026-08-09T00:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:47Z Disabled control of system clock
I20260809 00:37:47.468308 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:39421
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:39421
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36341/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:37:47.551767 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:47.551972 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:47.551993 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:47.553182 13561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:47.553228 13561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:47.553241 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:47.553253 13561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:47.553265 13561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:37:47.553277 13561 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:37:47.553288 13561 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36341/jwks.json
I20260809 00:37:47.554591 13561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:39421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36341/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:47.554791 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:47.555028 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:47.560263 13566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.560250 13567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.560227 13569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:47.560279 13561 server_base.cc:1034] running on GCE node
I20260809 00:37:47.560740 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:47.561034 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:47.562266 13561 hybrid_clock.cc:648] HybridClock initialized: now 1786235867562243 us; error 37 us; skew 500 ppm
I20260809 00:37:47.563473 13561 webserver.cc:460] Webserver started at http://127.7.105.126:40347/ using document root <none> and password file <none>
I20260809 00:37:47.563746 13561 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:47.563818 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:47.563971 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:47.565112 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "ecc8a8e92325feb4ec44b29a2181f9ec"
server_key_iv: "4d03b24665b4e51e9859a2ce1fa69f86"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.565475 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "ecc8a8e92325feb4ec44b29a2181f9ec"
server_key_iv: "4d03b24665b4e51e9859a2ce1fa69f86"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.566824 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:47.567538 13580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:47.567725 13561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:47.567834 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "ecc8a8e92325feb4ec44b29a2181f9ec"
server_key_iv: "4d03b24665b4e51e9859a2ce1fa69f86"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.567909 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:47.586758 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:47.587106 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:47.587267 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:47.591475 13561 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:39421
I20260809 00:37:47.591508 13642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:39421 every 8 connection(s)
I20260809 00:37:47.591863 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:47.592598 13643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:47.592904 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13561
I20260809 00:37:47.593066 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:47.593292 7589 external_mini_cluster.cc:1428] Setting key c6e282c3090fd49ec66e98b00babd3c6
I20260809 00:37:47.596521 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5: Bootstrap starting.
I20260809 00:37:47.597255 13643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:47.597589 13643 log.cc:824] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:47.598382 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5: No bootstrap required, opened a new log
I20260809 00:37:47.599597 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.593932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.7.105.126:39421)
0809 00:37:47.594160 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.594168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.594196 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:37:47.594828 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.594836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.594903 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.595037 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:47.595313 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.596397 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.597479 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.597966 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.598114 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.598884 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:47.598888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:47.598888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:47.599122 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:47.599156 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:47.599169 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:47.599254 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:47.599338 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 00:37:47.599424 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 00:37:47.600572 13643 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } }
I20260809 00:37:47.600740 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:47.600771 13643 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:47.600801 13643 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8dddcb7e4b34af9a48fb0c9f46367d5, State: Initialized, Role: FOLLOWER
I20260809 00:37:47.600932 13643 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } }
I20260809 00:37:47.601001 13643 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:47.601053 13643 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:47.601104 13643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:47.601781 13643 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } }
I20260809 00:37:47.601909 13643 leader_election.cc:302] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8dddcb7e4b34af9a48fb0c9f46367d5; no voters:
I20260809 00:37:47.602088 13643 leader_election.cc:288] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:47.602123 13649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:47.602236 13649 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [term 1 LEADER]: Becoming Leader. State: Replica: a8dddcb7e4b34af9a48fb0c9f46367d5, State: Running, Role: LEADER
I20260809 00:37:47.602321 13649 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } }
I20260809 00:37:47.602357 13643 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:47.602654 13651 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8dddcb7e4b34af9a48fb0c9f46367d5. Latest consensus state: current_term: 1 leader_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } } }
I20260809 00:37:47.602654 13650 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8dddcb7e4b34af9a48fb0c9f46367d5" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 39421 } } }
I20260809 00:37:47.602814 13650 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.602803 13651 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.603099 13656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:47.603645 13656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:47.605268 13656 catalog_manager.cc:1347] Generated new cluster ID: 14fac30dda454fc5b93e83a287efc6b6
I20260809 00:37:47.605336 13656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:47.630553 13656 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:47.631150 13656 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:47.638825 13656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8dddcb7e4b34af9a48fb0c9f46367d5: Generated new TSK 0
I20260809 00:37:47.639045 13656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:47.650373 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.646630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.7.105.126:39421)
0809 00:37:47.646701 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.646703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.646803 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:37:47.646847 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.646848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.646862 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.646952 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:37:47.647061 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.647786 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.648630 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.648953 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.649027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.649169 (+ 142us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:37:47.650152 (+ 983us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36341/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:37:47.650295 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36341/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 00:37:47.650442 13647 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36341/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 00:37:47.652266 13676 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:47.646546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:39421 (local address 127.0.0.1:40708)
0809 00:37:47.646732 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:47.646741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:47.646792 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:47.646949 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:47.646950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:47.646951 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:37:47.646996 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.646998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.647889 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.647890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.648544 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.648546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.649039 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.649040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.649082 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.649086 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:37:47.650264 (+ 1178us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36341/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:37:47.652167 (+ 1903us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.105.126:39421: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36341/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1705,"thread_start_us":115,"threads_started":1}
I20260809 00:37:47.652547 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13561
E20260809 00:37:47.665685 13679 webserver.cc:533] Webserver: sslize failed:
E20260809 00:37:47.665758 13679 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T00:37:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (237 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T00:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:47Z Disabled control of system clock
I20260809 00:37:47.703543 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:42371
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:42371
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/testchainca.cert with env {}
W20260809 00:37:47.785043 13684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:47.785295 13684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:47.785333 13684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:47.786515 13684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:47.786582 13684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:47.786603 13684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:47.786623 13684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:47.786656 13684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:37:47.786679 13684 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:37:47.786700 13684 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 00:37:47.788239 13684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:42371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:42371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:47.788518 13684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:47.788744 13684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:47.793782 13690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.793778 13693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:47.793761 13689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:47.793813 13684 server_base.cc:1034] running on GCE node
I20260809 00:37:47.794279 13684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:47.794519 13684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:47.795665 13684 hybrid_clock.cc:648] HybridClock initialized: now 1786235867795645 us; error 37 us; skew 500 ppm
I20260809 00:37:47.796947 13684 webserver.cc:460] Webserver started at http://127.7.105.126:36893/ using document root <none> and password file <none>
I20260809 00:37:47.797139 13684 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:47.797180 13684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:47.797268 13684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:47.798261 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "da526354f824439c9a965ffe02be9b36"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "fce4c70991c50f4693b3432e45dd29c1"
server_key_iv: "2564de0de57439ad9d334e50a637a385"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.798784 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "da526354f824439c9a965ffe02be9b36"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "fce4c70991c50f4693b3432e45dd29c1"
server_key_iv: "2564de0de57439ad9d334e50a637a385"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.800192 13684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:47.800895 13702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:47.801074 13684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:47.801163 13684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "da526354f824439c9a965ffe02be9b36"
format_stamp: "Formatted at 2026-08-09 00:37:47 on dist-test-slave-1zqn"
server_key: "fce4c70991c50f4693b3432e45dd29c1"
server_key_iv: "2564de0de57439ad9d334e50a637a385"
server_key_version: "encryptionkey@0"
I20260809 00:37:47.801218 13684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:47.812422 13684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:47.812698 13684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:47.812804 13684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:47.816776 13684 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:42371
I20260809 00:37:47.816836 13762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:42371 every 8 connection(s)
I20260809 00:37:47.817202 13684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:47.817904 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13684
I20260809 00:37:47.818017 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:47.818204 7589 external_mini_cluster.cc:1428] Setting key d6ceed23bbef256cb99969046ff703eb
I20260809 00:37:47.818375 13763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:47.821954 13763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: Bootstrap starting.
I20260809 00:37:47.822809 13763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:47.823168 13763 log.cc:824] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:47.824074 13763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: No bootstrap required, opened a new log
I20260809 00:37:47.824558 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.818777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.7.105.126:42371)
0809 00:37:47.819004 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.819015 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.819051 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:37:47.819609 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.819619 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.819709 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.819868 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:47.820025 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.821418 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.822270 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.822929 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.823065 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.823741 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:47.823745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:47.823745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:47.823956 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:47.823999 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:47.824013 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:47.824149 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:47.824239 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 00:37:47.824320 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260809 00:37:47.825655 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:47.826646 13763 raft_consensus.cc:348] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } }
I20260809 00:37:47.826823 13763 raft_consensus.cc:374] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:47.826864 13763 raft_consensus.cc:729] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da526354f824439c9a965ffe02be9b36, State: Initialized, Role: FOLLOWER
I20260809 00:37:47.826992 13763 consensus_queue.cc:260] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } }
I20260809 00:37:47.827062 13763 raft_consensus.cc:388] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:47.827085 13763 raft_consensus.cc:482] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:47.827117 13763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:47.827648 13763 raft_consensus.cc:504] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } }
I20260809 00:37:47.827735 13763 leader_election.cc:302] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da526354f824439c9a965ffe02be9b36; no voters:
I20260809 00:37:47.827865 13763 leader_election.cc:288] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:47.827947 13768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:47.828140 13768 raft_consensus.cc:686] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [term 1 LEADER]: Becoming Leader. State: Replica: da526354f824439c9a965ffe02be9b36, State: Running, Role: LEADER
I20260809 00:37:47.828147 13763 sys_catalog.cc:564] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:47.828249 13768 consensus_queue.cc:237] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } }
I20260809 00:37:47.828631 13770 sys_catalog.cc:455] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da526354f824439c9a965ffe02be9b36. Latest consensus state: current_term: 1 leader_uuid: "da526354f824439c9a965ffe02be9b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } } }
I20260809 00:37:47.828744 13770 sys_catalog.cc:458] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:47.828626 13769 sys_catalog.cc:455] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da526354f824439c9a965ffe02be9b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da526354f824439c9a965ffe02be9b36" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 42371 } } }
I20260809 00:37:47.828924 13769 sys_catalog.cc:458] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36 [sys.catalog]: This master's current role is: LEADER
W20260809 00:37:47.830078 13787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:47.830143 13787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:47.830227 13777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:47.830410 13777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:47.831924 13777 catalog_manager.cc:1347] Generated new cluster ID: eb0ce550f78e482e8c6e1d72d1435e75
I20260809 00:37:47.831979 13777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:47.857692 13777 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:47.858256 13777 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:47.868640 13777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da526354f824439c9a965ffe02be9b36: Generated new TSK 0
I20260809 00:37:47.868857 13777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:47.943331 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:47.880365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.7.105.126:42371)
0809 00:37:47.880432 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:47.880434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:47.880535 (+ 101us) server_negotiation.cc:407] Connection header received
0809 00:37:47.880626 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:47.880627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:47.880637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:47.880698 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:37:47.880851 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.881332 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.882466 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.882658 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.882707 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.882895 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:37:47.943111 (+ 60216us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:37:47.943269 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 00:37:47.943420 13766 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 00:37:47.943531 13798 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:47.880313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:42371 (local address 127.0.0.1:43572)
0809 00:37:47.880464 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:47.880476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:47.880552 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:47.880713 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:47.880714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:47.880716 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:37:47.880784 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.880786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.881423 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.881424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.882380 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:47.882383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:47.882730 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:47.882731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:47.882795 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:47.882802 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:37:47.943263 (+ 60461us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:37:47.943412 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.105.126:42371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
I20260809 00:37:47.943893 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13684
2026-08-09T00:37:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (269 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:48 dist-test-slave-1zqn krb5kdc[13814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:48 dist-test-slave-1zqn krb5kdc[13814](info): set up 2 sockets
Aug 09 00:37:48 dist-test-slave-1zqn krb5kdc[13814](info): commencing operation
krb5kdc: starting...
W20260809 00:37:49.365736 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 00:37:49.518973 7589 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.153s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 00:37:49.748482 7589 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.229s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 00:37:49.948681 7589 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.200s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/test-user.keytab.
W20260809 00:37:50.314949 7589 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.366s user 0.000s sys 0.002s
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:37:50.364913 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40239
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:50.450729 13831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:50.450943 13831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:50.450966 13831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:50.452136 13831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:50.452186 13831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:50.452201 13831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:50.452214 13831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:50.452226 13831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:50.453630 13831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:50.453831 13831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:50.454010 13831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:50.458956 13839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:50.458956 13841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:50.458998 13836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:50.459208 13831 server_base.cc:1034] running on GCE node
I20260809 00:37:50.459446 13831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:50.459756 13831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:50.460901 13831 hybrid_clock.cc:648] HybridClock initialized: now 1786235870460888 us; error 26 us; skew 500 ppm
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:50.467170 13831 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:37:50.467640 13831 webserver.cc:460] Webserver started at http://127.7.105.126:42213/ using document root <none> and password file <none>
I20260809 00:37:50.467835 13831 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:50.467877 13831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:50.467970 13831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:50.469097 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "8f2448eed44c59048530bd83740bacfe"
server_key_iv: "f35003fc3745119e5c3430e4430fca75"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.469410 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "8f2448eed44c59048530bd83740bacfe"
server_key_iv: "f35003fc3745119e5c3430e4430fca75"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.470743 13831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:50.471462 13850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.471640 13831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:50.471752 13831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "8f2448eed44c59048530bd83740bacfe"
server_key_iv: "f35003fc3745119e5c3430e4430fca75"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.471829 13831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:50.517074 13831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:50.519569 13831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:50.519745 13831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:50.523962 13831 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:40239
I20260809 00:37:50.524024 13912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:40239 every 8 connection(s)
I20260809 00:37:50.524459 13831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:50.525367 13913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:50.527766 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c: Bootstrap starting.
I20260809 00:37:50.528421 13913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:50.528728 13913 log.cc:824] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:50.529491 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c: No bootstrap required, opened a new log
I20260809 00:37:50.531039 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13831
I20260809 00:37:50.531189 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:50.531384 7589 external_mini_cluster.cc:1428] Setting key a50e62c4fe66732eaf1a97a95e2186d4
I20260809 00:37:50.531404 13913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } }
I20260809 00:37:50.531611 13913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:50.531641 13913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbc0ae610894aaeb0bd75b8c36c2d4c, State: Initialized, Role: FOLLOWER
I20260809 00:37:50.531833 13913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } }
I20260809 00:37:50.531924 13913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:50.532020 13913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:50.532147 13913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:50.533059 13913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } }
I20260809 00:37:50.533185 13913 leader_election.cc:302] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bbc0ae610894aaeb0bd75b8c36c2d4c; no voters:
I20260809 00:37:50.533385 13913 leader_election.cc:288] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:50.533416 13919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:50.533532 13919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [term 1 LEADER]: Becoming Leader. State: Replica: 9bbc0ae610894aaeb0bd75b8c36c2d4c, State: Running, Role: LEADER
I20260809 00:37:50.533669 13913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:50.533648 13919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } }
I20260809 00:37:50.534107 13920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } } }
I20260809 00:37:50.534178 13921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bbc0ae610894aaeb0bd75b8c36c2d4c. Latest consensus state: current_term: 1 leader_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbc0ae610894aaeb0bd75b8c36c2d4c" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40239 } } }
I20260809 00:37:50.534253 13920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:50.534348 13921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:50.534709 13929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:50.535250 13929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:50.537036 13929 catalog_manager.cc:1347] Generated new cluster ID: 1edd5f6c99234032bff4a363c874b463
I20260809 00:37:50.537091 13929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:50.542158 13929 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:50.542652 13929 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:50.543686 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:50.531918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.7.105.126:40239)
0809 00:37:50.532143 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:50.532154 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:50.532210 (+ 56us) server_negotiation.cc:407] Connection header received
0809 00:37:50.532346 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:50.532352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:50.532396 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:50.532534 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:50.533205 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.534658 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.536012 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.536639 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.536763 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:50.539428 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:50.539432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:50.539433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:50.539445 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:50.541377 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.541789 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.541792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.541793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.541889 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.542113 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.542114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.542114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.543023 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:50.543214 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:50.543291 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:37:50.543446 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":108,"threads_started":1}
I20260809 00:37:50.548599 13929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bbc0ae610894aaeb0bd75b8c36c2d4c: Generated new TSK 0
I20260809 00:37:50.548763 13929 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:37:50.579803 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:50.663864 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:50.664136 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:50.664180 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:50.665354 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:50.665417 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:50.665463 13944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:50.666957 13944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:50.667223 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:50.667459 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:50.672662 13951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:50.672662 13953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:50.672698 13950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:50.672775 13944 server_base.cc:1034] running on GCE node
I20260809 00:37:50.673179 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:50.673480 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:50.674652 13944 hybrid_clock.cc:648] HybridClock initialized: now 1786235870674630 us; error 45 us; skew 500 ppm
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:50.677474 13944 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:37:50.677927 13944 webserver.cc:460] Webserver started at http://127.7.105.65:40889/ using document root <none> and password file <none>
I20260809 00:37:50.678114 13944 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:50.678156 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:50.678247 13944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:50.679250 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "cf7af460203b450ca889a267def5fd13"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "a4a5125cc7a8c7cfbcf8ad1ca0ab6621"
server_key_iv: "e7b40a363f38b95a5d43ffac41caa48f"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.679560 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "cf7af460203b450ca889a267def5fd13"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "a4a5125cc7a8c7cfbcf8ad1ca0ab6621"
server_key_iv: "e7b40a363f38b95a5d43ffac41caa48f"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.680944 13944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:50.681653 13963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.681834 13944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:50.681926 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "cf7af460203b450ca889a267def5fd13"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "a4a5125cc7a8c7cfbcf8ad1ca0ab6621"
server_key_iv: "e7b40a363f38b95a5d43ffac41caa48f"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.682001 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:50.697568 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:50.699968 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:50.700166 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:50.700415 13944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:50.700740 13944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:50.700774 13944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.700816 13944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:50.700842 13944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.706902 13944 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35343
I20260809 00:37:50.706961 14092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35343 every 8 connection(s)
I20260809 00:37:50.707273 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:50.715786 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 13944
I20260809 00:37:50.715927 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:50.716207 7589 external_mini_cluster.cc:1428] Setting key 8e8f3876ed82ede596d287368a814c0b
I20260809 00:37:50.716856 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:50.707946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:42133 (local address 127.7.105.126:40239)
0809 00:37:50.708048 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:50.708072 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:50.708263 (+ 191us) server_negotiation.cc:407] Connection header received
0809 00:37:50.709177 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:50.709178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:50.709189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:50.709241 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:50.709810 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.710310 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.711279 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.711475 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.711526 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:50.713633 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:50.713635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:50.713635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:50.713642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:50.715081 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.715583 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.715583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.715584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.715638 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.716038 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.716039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.716039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.716207 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:50.716295 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:50.716662 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 00:37:50.716761 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:37:50.717051 14095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:50.707900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40239 (local address 127.7.105.65:42133)
0809 00:37:50.708119 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:50.708172 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:50.709083 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:50.709298 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:50.709301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:50.709547 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:50.709738 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:50.709742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.710388 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.710389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:50.711201 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:50.711203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.711548 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.711550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:50.711650 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:50.712145 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:50.712155 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:50.713536 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:50.715180 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:50.715189 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:50.715192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:50.715466 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:50.715713 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:50.715714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:50.715714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:50.715876 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:50.716288 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:50.716297 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:50.716500 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 00:37:50.716634 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 00:37:50.716792 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":99,"threads_started":1}
I20260809 00:37:50.717527 14093 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40239
I20260809 00:37:50.717625 14093 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:50.717831 14093 heartbeater.cc:511] Master 127.7.105.126:40239 requested a full tablet report, sending...
I20260809 00:37:50.718365 13870 ts_manager.cc:194] Registered new tserver with Master: cf7af460203b450ca889a267def5fd13 (127.7.105.65:35343)
I20260809 00:37:50.719151 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:42133
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:37:50.743149 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:50.827008 14099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:50.827234 14099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:50.827258 14099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:50.828474 14099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:50.828516 14099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:50.828549 14099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:50.830060 14099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:50.830305 14099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:50.830564 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:50.835875 14107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:50.835932 14106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:50.835911 14099 server_base.cc:1034] running on GCE node
W20260809 00:37:50.835892 14109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:50.836344 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:50.836619 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:50.837798 14099 hybrid_clock.cc:648] HybridClock initialized: now 1786235870837790 us; error 35 us; skew 500 ppm
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:50.840503 14099 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:37:50.840937 14099 webserver.cc:460] Webserver started at http://127.7.105.66:39901/ using document root <none> and password file <none>
I20260809 00:37:50.841121 14099 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:50.841161 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:50.841254 14099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:50.842252 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "7e8446723d96473aa5945cb52e9f8506"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "204b9bf86cfe921880b22c282c9c37ea"
server_key_iv: "e74bcd9e58904103eecacfa97c4c8058"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.842576 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "7e8446723d96473aa5945cb52e9f8506"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "204b9bf86cfe921880b22c282c9c37ea"
server_key_iv: "e74bcd9e58904103eecacfa97c4c8058"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.843880 14099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:50.844691 14117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.844857 14099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.844965 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "7e8446723d96473aa5945cb52e9f8506"
format_stamp: "Formatted at 2026-08-09 00:37:50 on dist-test-slave-1zqn"
server_key: "204b9bf86cfe921880b22c282c9c37ea"
server_key_iv: "e74bcd9e58904103eecacfa97c4c8058"
server_key_version: "encryptionkey@0"
I20260809 00:37:50.845045 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:50.880584 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:50.883087 14099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:50.883261 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:50.883529 14099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:50.883849 14099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:50.883911 14099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.883980 14099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:50.884011 14099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:50.890072 14099 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:44315
I20260809 00:37:50.890125 14247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:44315 every 8 connection(s)
I20260809 00:37:50.890453 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:37:50 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:50.898854 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14099
I20260809 00:37:50.898970 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:50.899180 7589 external_mini_cluster.cc:1428] Setting key 0a61b1d246d4b832aa98060206b61dc0
I20260809 00:37:50.900242 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:50.891177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:39745 (local address 127.7.105.126:40239)
0809 00:37:50.891221 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:50.891223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:50.891453 (+ 230us) server_negotiation.cc:407] Connection header received
0809 00:37:50.892468 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:50.892469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:50.892480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:50.892534 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:50.893126 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.893621 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.894743 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.894955 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.895032 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:50.897233 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:50.897235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:50.897235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:50.897243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:50.898677 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.899158 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.899159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.899160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.899222 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:50.899502 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:50.899503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:50.899503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:50.899638 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:50.899702 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:50.900050 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 00:37:50.900177 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:50.900533 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:50.891173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40239 (local address 127.7.105.66:39745)
0809 00:37:50.891329 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:50.891364 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:50.892370 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:50.892591 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:50.892595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:50.892851 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:50.893060 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:50.893064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.893703 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.893704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:50.894623 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:50.894626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:50.895038 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:50.895040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:50.895166 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:50.895721 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:50.895729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:50.897131 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:50.898780 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:50.898794 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:50.898797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:50.899078 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:50.899295 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:50.899295 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:50.899296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:50.899440 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:50.899713 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:50.899719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:50.899892 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 00:37:50.900033 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 00:37:50.900258 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260809 00:37:50.901075 14248 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40239
I20260809 00:37:50.901189 14248 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:50.901386 14248 heartbeater.cc:511] Master 127.7.105.126:40239 requested a full tablet report, sending...
I20260809 00:37:50.901747 13870 ts_manager.cc:194] Registered new tserver with Master: 7e8446723d96473aa5945cb52e9f8506 (127.7.105.66:44315)
I20260809 00:37:50.902172 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:39745
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:37:50.927301 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:51.013922 14254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:51.014199 14254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:51.014245 14254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:51.015458 14254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:51.015534 14254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:51.015580 14254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:51.017184 14254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:51.017572 14254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:51.017838 14254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:37:51.023478 14254 server_base.cc:1034] running on GCE node
W20260809 00:37:51.023614 14260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:51.023620 14264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:51.023643 14261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:51.024017 14254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:51.024304 14254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:51.025465 14254 hybrid_clock.cc:648] HybridClock initialized: now 1786235871025455 us; error 33 us; skew 500 ppm
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.028317 14254 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:37:51.028767 14254 webserver.cc:460] Webserver started at http://127.7.105.67:38253/ using document root <none> and password file <none>
I20260809 00:37:51.029022 14254 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:51.029100 14254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:51.029275 14254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:51.030354 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "f1a31c22388c4fecbac1e6ec0efafb74"
format_stamp: "Formatted at 2026-08-09 00:37:51 on dist-test-slave-1zqn"
server_key: "40fbd76e34a5941bf9c0600177916b53"
server_key_iv: "cab5b61658a16df26450a15b76fe3274"
server_key_version: "encryptionkey@0"
I20260809 00:37:51.030740 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "f1a31c22388c4fecbac1e6ec0efafb74"
format_stamp: "Formatted at 2026-08-09 00:37:51 on dist-test-slave-1zqn"
server_key: "40fbd76e34a5941bf9c0600177916b53"
server_key_iv: "cab5b61658a16df26450a15b76fe3274"
server_key_version: "encryptionkey@0"
I20260809 00:37:51.032317 14254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:51.033123 14272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:51.033331 14254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:51.033454 14254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "f1a31c22388c4fecbac1e6ec0efafb74"
format_stamp: "Formatted at 2026-08-09 00:37:51 on dist-test-slave-1zqn"
server_key: "40fbd76e34a5941bf9c0600177916b53"
server_key_iv: "cab5b61658a16df26450a15b76fe3274"
server_key_version: "encryptionkey@0"
I20260809 00:37:51.033535 14254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:51.044147 14254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:51.046321 14254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:51.046576 14254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:51.046859 14254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:51.047212 14254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:51.047250 14254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:51.047293 14254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:51.047320 14254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:51.053541 14254 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:40289
I20260809 00:37:51.053618 14404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:40289 every 8 connection(s)
I20260809 00:37:51.053956 14254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:37:51.063153 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:51.054696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:48971 (local address 127.7.105.126:40239)
0809 00:37:51.054771 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.054773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.054982 (+ 209us) server_negotiation.cc:407] Connection header received
0809 00:37:51.055992 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.055994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.056006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.056222 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.056880 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.057362 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.058334 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.058514 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.058573 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:51.060783 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:51.060784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:51.060785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:51.060791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:51.061997 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.062363 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.062364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.062364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.062414 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.062640 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.062640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.062640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.062741 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:51.062791 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:51.063017 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 00:37:51.063086 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:37:51.063354 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14254
I20260809 00:37:51.063402 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:51.054689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40239 (local address 127.7.105.67:48971)
0809 00:37:51.054850 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:51.054893 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:51.055836 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:51.056297 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:51.056301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:51.056579 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:51.056779 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:51.056784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.057443 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.057444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:51.058259 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:51.058261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.058589 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.058589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:51.058699 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:51.059252 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:51.059261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:51.060704 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:51.062075 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:51.062081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:51.062083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:51.062296 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:51.062472 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:51.062473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:51.062473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:51.062585 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:51.062795 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:51.062799 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:51.062912 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:37:51.062992 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 00:37:51.063113 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260809 00:37:51.063449 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:51.063619 7589 external_mini_cluster.cc:1428] Setting key 6ad1fd441e8fbe31d3ea4a2b5dbb4179
I20260809 00:37:51.064038 14405 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40239
I20260809 00:37:51.064188 14405 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:51.064381 14405 heartbeater.cc:511] Master 127.7.105.126:40239 requested a full tablet report, sending...
I20260809 00:37:51.064787 13868 ts_manager.cc:194] Registered new tserver with Master: f1a31c22388c4fecbac1e6ec0efafb74 (127.7.105.67:40289)
I20260809 00:37:51.065182 13868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:48971
I20260809 00:37:51.075587 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/test-admin.keytab.
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.093747 7589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 00:37:51.095494 14416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:51.094621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.7.105.65:35343)
0809 00:37:51.094814 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.094819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.094828 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:51.094872 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.094874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.094887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.094953 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.095404 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52370: BlockingRecv error: recv got EOF from 127.0.0.1:52370 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
W20260809 00:37:51.095710 7589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.096798 7589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.097740 7589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:51.105990 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:51.098327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.7.105.65:35343)
0809 00:37:51.098389 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.098391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.098505 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:37:51.098543 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.098544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.098557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.098608 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.098991 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.099752 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.100527 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.100852 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.100909 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:51.102987 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:51.102995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:51.102996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:51.103002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:51.104678 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.105043 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.105047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.105047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.105114 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.105327 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.105328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.105329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.105490 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:51.105613 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:51.105673 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:37:51.105744 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 00:37:51.107987 14416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:51.107464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.7.105.65:35343)
0809 00:37:51.107498 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.107500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.107540 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:37:51.107589 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.107590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.107598 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.107646 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.107914 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52394: BlockingRecv error: recv got EOF from 127.0.0.1:52394 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 00:37:51.108110 7589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.109107 7589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.109942 7589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:51.116990 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:51.110419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.7.105.65:35343)
0809 00:37:51.110474 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.110476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.110497 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:37:51.110560 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.110560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.110570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.110613 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.110921 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.111449 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.112173 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.112372 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.112425 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:51.114630 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:51.114632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:51.114632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:51.114644 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:51.115880 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.116327 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.116328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.116329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.116387 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.116551 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.116552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.116552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.116643 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:51.116716 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:51.116799 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:37:51.116866 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 00:37:51.118945 14416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:51.118268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.7.105.65:35343)
0809 00:37:51.118334 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.118336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.118342 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:51.118358 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.118358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.118366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.118433 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.118894 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52418: BlockingRecv error: recv got EOF from 127.0.0.1:52418 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 00:37:51.119143 7589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235843038573-7589-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.120479 7589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:51.121475 7589 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:37:51.129055 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:51.122066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.7.105.65:35343)
0809 00:37:51.122123 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:51.122125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:51.122134 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:51.122212 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:51.122213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:51.122221 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:51.122268 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:51.122693 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.123157 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.123934 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:51.124192 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:51.124266 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:51.126413 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:51.126414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:51.126414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:51.126421 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:51.127802 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.128379 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.128380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.128380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.128434 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:51.128624 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:51.128624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:51.128625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:51.128713 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:51.128783 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:51.128889 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:37:51.128943 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:51.129783 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13944
I20260809 00:37:51.137252 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14099
I20260809 00:37:51.146126 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14254
I20260809 00:37:51.151968 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 13831
2026-08-09T00:37:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3025 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[14435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[14435](info): set up 2 sockets
Aug 09 00:37:51 dist-test-slave-1zqn krb5kdc[14435](info): commencing operation
krb5kdc: starting...
W20260809 00:37:52.264127 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab.
Entry for principal oryx/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab.
I20260809 00:37:52.395841 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43213
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab
--principal=oryx/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:52.484436 14450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:52.484714 14450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:52.484751 14450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:52.485922 14450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:52.485999 14450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:52.486027 14450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:52.486060 14450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:52.486086 14450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:52.487609 14450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab
--principal=oryx/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:52.487903 14450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:52.488210 14450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:52.493438 14456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.493462 14459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.493616 14457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:52.493680 14450 server_base.cc:1034] running on GCE node
I20260809 00:37:52.494024 14450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:52.494298 14450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:52.495463 14450 hybrid_clock.cc:648] HybridClock initialized: now 1786235872495442 us; error 36 us; skew 500 ppm
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:52.498208 14450 init.cc:377] Logged in from keytab as oryx/127.7.105.126@KRBTEST.COM (short username oryx)
I20260809 00:37:52.498696 14450 webserver.cc:460] Webserver started at http://127.7.105.126:42603/ using document root <none> and password file <none>
I20260809 00:37:52.498890 14450 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:52.498932 14450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:52.499029 14450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:52.500149 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "0a5aa01c47434b4586800d33a846ca77"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "17a484f7b88829a3691ef95c54f2881b"
server_key_iv: "feb419b20a5815a9b61a9d3b74f6e9f4"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.500485 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "0a5aa01c47434b4586800d33a846ca77"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "17a484f7b88829a3691ef95c54f2881b"
server_key_iv: "feb419b20a5815a9b61a9d3b74f6e9f4"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.501786 14450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:52.502532 14466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.502753 14450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:52.502863 14450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "0a5aa01c47434b4586800d33a846ca77"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "17a484f7b88829a3691ef95c54f2881b"
server_key_iv: "feb419b20a5815a9b61a9d3b74f6e9f4"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.502942 14450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:52.521088 14450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:52.523334 14450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:52.523490 14450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:52.527941 14450 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:43213
I20260809 00:37:52.528010 14529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:43213 every 8 connection(s)
I20260809 00:37:52.528420 14450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:52.529595 14530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:52.532089 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14450
I20260809 00:37:52.532232 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:52.532341 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Bootstrap starting.
I20260809 00:37:52.532466 7589 external_mini_cluster.cc:1428] Setting key 3d8eaedd92a203894334d3767ed8a231
I20260809 00:37:52.533162 14530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:52.533515 14530 log.cc:824] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:52.534410 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: No bootstrap required, opened a new log
I20260809 00:37:52.536820 14530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } }
I20260809 00:37:52.537007 14530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:52.537047 14530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5aa01c47434b4586800d33a846ca77, State: Initialized, Role: FOLLOWER
I20260809 00:37:52.537235 14530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } }
I20260809 00:37:52.537345 14530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:52.537403 14530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:52.537461 14530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:52.538220 14530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } }
I20260809 00:37:52.538357 14530 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5aa01c47434b4586800d33a846ca77; no voters:
I20260809 00:37:52.538610 14530 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:52.538638 14536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:52.538800 14536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 LEADER]: Becoming Leader. State: Replica: 0a5aa01c47434b4586800d33a846ca77, State: Running, Role: LEADER
I20260809 00:37:52.538902 14536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } }
I20260809 00:37:52.538945 14530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:52.539281 14536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } }
I20260809 00:37:52.539376 14536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:52.539531 14536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5aa01c47434b4586800d33a846ca77. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } }
I20260809 00:37:52.539685 14536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:52.539897 14541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:52.540632 14541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:52.542728 14541 catalog_manager.cc:1347] Generated new cluster ID: 0f35bb16ac2c4ea5b5cf1f077046c016
I20260809 00:37:52.542793 14541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:52.545387 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:52.533046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.7.105.126:43213)
0809 00:37:52.533249 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:52.533258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:52.533296 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:37:52.533887 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:52.533897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:52.533963 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:52.534146 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:52.535132 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.535995 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.537169 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.537725 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.537810 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:52.541037 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:52.541041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:52.541042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:52.541055 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:52.543615 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.544040 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.544044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.544044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.544205 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.544422 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.544423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.544423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.544755 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:52.544929 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:52.545009 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:37:52.545157 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260809 00:37:52.567826 14541 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:52.568655 14541 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:52.573868 14541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Generated new TSK 0
I20260809 00:37:52.574076 14541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
I20260809 00:37:52.605103 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:52.694938 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:52.695174 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:52.695214 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:52.696440 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:52.696503 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:52.696552 14561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:52.698138 14561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:52.698422 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:52.698673 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:52.703995 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.704133 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.704003 14572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:52.704278 14561 server_base.cc:1034] running on GCE node
I20260809 00:37:52.704493 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:52.704716 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:52.705852 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786235872705840 us; error 30 us; skew 500 ppm
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:52.708427 14561 init.cc:377] Logged in from keytab as oryx/127.7.105.65@KRBTEST.COM (short username oryx)
I20260809 00:37:52.708892 14561 webserver.cc:460] Webserver started at http://127.7.105.65:43107/ using document root <none> and password file <none>
I20260809 00:37:52.709141 14561 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:52.709211 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:52.709362 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:52.710566 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "217efa1ae2894d0592fa498828bbd9d6"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "65ee0ebc9b806139d872954bd9fc22e2"
server_key_iv: "af2de1fcfab657b86b28baf214a9933e"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.710938 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "217efa1ae2894d0592fa498828bbd9d6"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "65ee0ebc9b806139d872954bd9fc22e2"
server_key_iv: "af2de1fcfab657b86b28baf214a9933e"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.712388 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:52.713258 14581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.713547 14561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:52.713670 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "217efa1ae2894d0592fa498828bbd9d6"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "65ee0ebc9b806139d872954bd9fc22e2"
server_key_iv: "af2de1fcfab657b86b28baf214a9933e"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.713748 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:52.735373 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:52.737501 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:52.737685 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:52.737942 14561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:52.738332 14561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:52.738385 14561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.738432 14561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:52.738476 14561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.745019 14561 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35213
I20260809 00:37:52.745078 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35213 every 8 connection(s)
I20260809 00:37:52.745419 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:52.751678 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14561
I20260809 00:37:52.751797 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:52.751992 7589 external_mini_cluster.cc:1428] Setting key 4fc42496b1aa4b13f258bf61f3d608c8
I20260809 00:37:52.755707 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:52.746219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:39111 (local address 127.7.105.126:43213)
0809 00:37:52.746267 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:52.746269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:52.746507 (+ 238us) server_negotiation.cc:407] Connection header received
0809 00:37:52.747001 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:52.747002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:52.747013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:52.747070 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:52.748100 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.748638 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.749632 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.749841 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.749906 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:52.752341 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:52.752343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:52.752343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:52.752352 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:52.754079 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.754521 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.754523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.754523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.754597 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.754969 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.754970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.754971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.755095 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:52.755153 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:52.755548 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 00:37:52.755637 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:52.756029 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:52.746220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:39111)
0809 00:37:52.746374 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:52.746410 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:52.746895 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:52.747134 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:52.747137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:52.747730 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:52.747940 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:52.747945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.748729 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.748729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:52.749525 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:52.749527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.749926 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.749927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:52.750031 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:52.750541 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:52.750570 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:52.752217 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:52.754171 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:52.754179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:52.754181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:52.754439 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:52.754698 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:52.754699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:52.754700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:52.754884 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:52.755171 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:52.755178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:52.755362 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 00:37:52.755535 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 00:37:52.755744 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260809 00:37:52.756651 14715 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:52.756755 14715 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:52.756968 14715 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
I20260809 00:37:52.757611 14486 ts_manager.cc:194] Registered new tserver with Master: 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
I20260809 00:37:52.758250 14486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:39111
WARNING: no policy specified for oryx/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
I20260809 00:37:52.783171 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:52.867858 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:52.868142 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:52.868191 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:52.869424 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:52.869496 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:52.869541 14721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:52.871006 14721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:52.871296 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:52.871546 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:52.877135 14731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.877203 14728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:52.877188 14727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:52.877321 14721 server_base.cc:1034] running on GCE node
I20260809 00:37:52.877693 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:52.877977 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:52.879184 14721 hybrid_clock.cc:648] HybridClock initialized: now 1786235872879156 us; error 38 us; skew 500 ppm
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:52.882229 14721 init.cc:377] Logged in from keytab as oryx/127.7.105.66@KRBTEST.COM (short username oryx)
I20260809 00:37:52.882743 14721 webserver.cc:460] Webserver started at http://127.7.105.66:34455/ using document root <none> and password file <none>
I20260809 00:37:52.883023 14721 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:52.883101 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:52.883266 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:52.884562 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "942f1a77b5f748eb997b6acd4ed2308b"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "42900b91e6e7387c665196b304d8c636"
server_key_iv: "aa4d65cf2efd7c6aea6f1c059e80be61"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.884965 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "942f1a77b5f748eb997b6acd4ed2308b"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "42900b91e6e7387c665196b304d8c636"
server_key_iv: "aa4d65cf2efd7c6aea6f1c059e80be61"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.886512 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:52.887435 14739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.887678 14721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:52.887804 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "942f1a77b5f748eb997b6acd4ed2308b"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "42900b91e6e7387c665196b304d8c636"
server_key_iv: "aa4d65cf2efd7c6aea6f1c059e80be61"
server_key_version: "encryptionkey@0"
I20260809 00:37:52.887933 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:52.914738 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:52.917753 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:52.917964 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:52.918259 14721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:52.918655 14721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:52.918718 14721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.918766 14721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:52.918798 14721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:52.925381 14721 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35505
I20260809 00:37:52.925432 14863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35505 every 8 connection(s)
I20260809 00:37:52.925802 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:52.929334 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14721
I20260809 00:37:52.929469 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:52.929677 7589 external_mini_cluster.cc:1428] Setting key 68ba21bbcccd12564c7bbc992ef2ec1c
Aug 09 00:37:52 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:52.936622 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:52.926586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37109 (local address 127.7.105.126:43213)
0809 00:37:52.926653 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:52.926655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:52.926851 (+ 196us) server_negotiation.cc:407] Connection header received
0809 00:37:52.927384 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:52.927386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:52.927400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:52.927466 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:52.928554 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.929105 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.930545 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.930841 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.930919 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:52.933559 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:52.933561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:52.933561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:52.933569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:52.935147 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.935567 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.935569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.935569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.935637 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:52.935903 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:52.935904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:52.935905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:52.936031 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:52.936145 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:52.936425 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 00:37:52.936512 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:52.936825 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:52.926557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:37109)
0809 00:37:52.926737 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:52.926778 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:52.927258 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:52.927520 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:52.927523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:52.928228 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:52.928439 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:52.928461 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.929220 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.929221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:52.930442 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:52.930445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:52.931008 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:52.931009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:52.931143 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:52.931790 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:52.931800 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:52.933450 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:52.935231 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:52.935240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:52.935242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:52.935479 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:52.935718 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:52.935718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:52.935719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:52.935830 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:52.936136 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:52.936144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:52.936290 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 00:37:52.936403 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 00:37:52.936572 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260809 00:37:52.938037 14864 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:52.938133 14864 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:52.938344 14864 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
I20260809 00:37:52.938848 14486 ts_manager.cc:194] Registered new tserver with Master: 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505)
I20260809 00:37:52.939458 14486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:37109
WARNING: no policy specified for oryx/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
I20260809 00:37:52.959296 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:53.045567 14870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:53.045810 14870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:53.045852 14870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:53.047039 14870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:53.047109 14870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:53.047158 14870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:53.048743 14870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:53.049012 14870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:53.049257 14870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.054741 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.054750 14877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.054783 14870 server_base.cc:1034] running on GCE node
W20260809 00:37:53.054750 14875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.055217 14870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.055475 14870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.056631 14870 hybrid_clock.cc:648] HybridClock initialized: now 1786235873056616 us; error 30 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.059381 14870 init.cc:377] Logged in from keytab as oryx/127.7.105.67@KRBTEST.COM (short username oryx)
I20260809 00:37:53.059890 14870 webserver.cc:460] Webserver started at http://127.7.105.67:41495/ using document root <none> and password file <none>
I20260809 00:37:53.060181 14870 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.060253 14870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.060416 14870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:53.061559 14870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "b4622e06bd874a13bcfe35bc50ee7087"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "e5441d52f771cd8da24d7521ef7752f6"
server_key_iv: "dee2b99d365cf8b5f4812a2dd322b3d4"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.061951 14870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "b4622e06bd874a13bcfe35bc50ee7087"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "e5441d52f771cd8da24d7521ef7752f6"
server_key_iv: "dee2b99d365cf8b5f4812a2dd322b3d4"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.063450 14870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:53.064263 14886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.064440 14870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:53.064555 14870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "b4622e06bd874a13bcfe35bc50ee7087"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "e5441d52f771cd8da24d7521ef7752f6"
server_key_iv: "dee2b99d365cf8b5f4812a2dd322b3d4"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.064630 14870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.079244 14870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.081748 14870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:53.081918 14870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.082167 14870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:53.082504 14870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:53.082559 14870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.082607 14870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:53.082630 14870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.089407 14870 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38267
I20260809 00:37:53.089454 15017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38267 every 8 connection(s)
I20260809 00:37:53.089785 14870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:53.095057 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 14870
I20260809 00:37:53.095163 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:53.095336 7589 external_mini_cluster.cc:1428] Setting key cf6e3778dd5be7a788675f0bc55d78dc
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:53.099579 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.090534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:47837 (local address 127.7.105.126:43213)
0809 00:37:53.090608 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.090610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.090851 (+ 241us) server_negotiation.cc:407] Connection header received
0809 00:37:53.091479 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.091480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.091492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.091546 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.092635 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.093162 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.094237 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.094437 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.094497 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.097000 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.097002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.097002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.097009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.098337 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.098728 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.098729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.098729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.098781 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.098993 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.098993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.098994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.099097 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.099147 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.099386 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.099499 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:37:53.099642 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.090525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:47837)
0809 00:37:53.090720 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.090759 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.091380 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.091602 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.091606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.092323 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.092534 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.092539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.093255 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.093256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.094149 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.094151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.094526 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.094527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.094630 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.095218 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.095230 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.096920 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.098412 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.098418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.098420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.098660 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.098841 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.098842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.098842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.098939 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.099151 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.099155 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.099269 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.099350 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.099472 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260809 00:37:53.100186 15019 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:53.100337 15019 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.100553 15019 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
I20260809 00:37:53.100980 14486 ts_manager.cc:194] Registered new tserver with Master: b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
I20260809 00:37:53.101372 14486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:47837
I20260809 00:37:53.107751 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.105.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.125.keytab.
Entry for principal oryx/127.7.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.125.keytab.
I20260809 00:37:53.138903 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:38399
--webserver_interface=127.7.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.125.keytab
--principal=oryx/127.7.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43213,127.7.105.125:38399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:53.225432 15025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:53.225663 15025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:53.225689 15025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:53.226883 15025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:53.226933 15025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:53.226949 15025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:53.226965 15025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:53.226979 15025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:53.228762 15025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43213,127.7.105.125:38399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.125.keytab
--principal=oryx/127.7.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:38399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
W20260809 00:37:53.228919 15025 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.105.126:43213,127.7.105.125:38399'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 00:37:53.229082 15025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:53.229336 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.234468 15035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.234431 15031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.234432 15032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.234774 15025 server_base.cc:1034] running on GCE node
I20260809 00:37:53.235106 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.235422 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.236608 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786235873236590 us; error 45 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.239557 15025 init.cc:377] Logged in from keytab as oryx/127.7.105.125@KRBTEST.COM (short username oryx)
I20260809 00:37:53.240141 15025 webserver.cc:460] Webserver started at http://127.7.105.125:34617/ using document root <none> and password file <none>
I20260809 00:37:53.240406 15025 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.240485 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.240649 15025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:53.241842 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/instance:
uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "443e9684f110b7020caedb049f66017c"
server_key_iv: "5daea1eaa7b77fad2035a5155fb4720c"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.242214 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal/instance:
uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "443e9684f110b7020caedb049f66017c"
server_key_iv: "5daea1eaa7b77fad2035a5155fb4720c"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.243665 15025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:53.244640 15042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.244841 15025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:53.244937 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "443e9684f110b7020caedb049f66017c"
server_key_iv: "5daea1eaa7b77fad2035a5155fb4720c"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.244997 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.265055 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.267210 15025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:53.267364 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.271519 15025 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.125:38399
I20260809 00:37:53.271560 15102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.125:38399 every 8 connection(s)
I20260809 00:37:53.271946 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
I20260809 00:37:53.273020 15103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:53.274645 15103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:53.275287 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 15025
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.125@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
I20260809 00:37:53.284854 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.275418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.7.105.126:43213)
0809 00:37:53.275498 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.275501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.275827 (+ 326us) server_negotiation.cc:407] Connection header received
0809 00:37:53.276578 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.276580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.276592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.276648 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.277200 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.277720 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.278803 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.279102 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.279172 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.281680 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.281681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.281682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.281690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.283269 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.283813 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.283814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.283814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.283868 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.284160 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.284161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.284162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.284308 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.284364 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.284680 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.284768 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:37:53.285061 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.275385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58342)
0809 00:37:53.275687 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.275737 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.276432 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.276683 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.276685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.276948 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.277106 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.277110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.277814 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.277815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.278723 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.278726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.279184 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.279185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.279321 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.279895 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.279906 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.281586 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.283354 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.283360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.283363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.283730 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.283938 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.283939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.283939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.284097 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.284375 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.284380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.284530 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.284607 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.284757 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260809 00:37:53.285697 15103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:53.285774 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.275831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.7.105.125:38399)
0809 00:37:53.275964 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.275969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.276022 (+ 53us) server_negotiation.cc:407] Connection header received
0809 00:37:53.276435 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.276444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.276471 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.276571 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.277273 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.278274 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.279148 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.279589 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.279667 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.282251 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.282259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.282259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.282274 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.284333 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.284738 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.284741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.284742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.284800 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.285030 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.285032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.285032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.285242 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.285395 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.285505 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.285610 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.125@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
I20260809 00:37:53.291859 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.285961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37442)
0809 00:37:53.286042 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.286051 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.286114 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.286223 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.286224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.286358 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.286397 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.286398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.287044 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.287045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.287506 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.287507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.287832 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.287833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.287873 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.288250 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.288254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.289367 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.290963 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.290963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.290964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.291116 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.291284 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.291285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.291285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.291333 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.291510 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.291511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.291544 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.291639 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.291740 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 00:37:53.291919 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.286015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.7.105.125:38399)
0809 00:37:53.286047 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.286049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.286100 (+ 51us) server_negotiation.cc:407] Connection header received
0809 00:37:53.286165 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.286165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.286174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.286218 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.286472 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.286950 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.287583 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.287770 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.287819 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.289442 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.289443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.289443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.289449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.290894 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.291180 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.291180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.291181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.291227 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.291370 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.291370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.291371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.291450 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.291493 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.291680 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.291802 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":15}
I20260809 00:37:53.293236 15103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Bootstrap starting.
I20260809 00:37:53.294092 15103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:53.294389 15103 log.cc:824] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:53.295105 15103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: No bootstrap required, opened a new log
I20260809 00:37:53.296588 15103 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } }
I20260809 00:37:53.296710 15103 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:53.296731 15103 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff0a616e88fa4dc69d8bce30e7d1f82c, State: Initialized, Role: FOLLOWER
I20260809 00:37:53.296890 15103 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } }
I20260809 00:37:53.298094 15103 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:53.298103 15109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } } }
I20260809 00:37:53.298295 15109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:37:53.299901 15120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:53.299968 15120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:53.301853 15025 master_runner.cc:428] Detected that this master ff0a616e88fa4dc69d8bce30e7d1f82c is joining an existing cluster
I20260809 00:37:53.301956 15025 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.105.125:38399
I20260809 00:37:53.311256 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.303158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.7.105.126:43213)
0809 00:37:53.303342 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.303345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.303608 (+ 263us) server_negotiation.cc:407] Connection header received
0809 00:37:53.303646 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.303647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.303660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.303726 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.304220 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.304808 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.305958 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.306272 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.306374 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.308144 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.308146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.308147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.308156 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.309918 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310426 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.310427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.310427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.310489 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310695 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.310696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.310697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.310837 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.310903 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.311076 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.311158 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 00:37:53.311265 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.303655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.7.105.125:38399)
0809 00:37:53.303715 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.303717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.303904 (+ 187us) server_negotiation.cc:407] Connection header received
0809 00:37:53.303962 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.303962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.303971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.304025 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.304436 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.305185 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.306128 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.306409 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.306470 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.307821 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.307823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.307823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.307830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.309787 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310344 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.310346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.310346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.310406 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310608 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.310609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.310609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.310748 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.310838 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.310990 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.311141 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":43}
I20260809 00:37:53.311503 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.303711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37454)
0809 00:37:53.303818 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.303827 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.303905 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.304030 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.304030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.304277 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.304347 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.304349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.305268 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.305269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.306056 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.306058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.306513 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.306514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.306579 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.307097 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.307101 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.307723 (+ 622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.309888 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.309889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.309890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.310263 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.310481 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.310482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.310548 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.310828 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.310829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.310879 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.310952 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.311350 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260809 00:37:53.312119 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.303315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58348)
0809 00:37:53.303518 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.303530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.303596 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.303779 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.303781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.304014 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.304132 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.304134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.304890 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.304892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.305864 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.305866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.306367 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.306368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.306429 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.307012 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.307021 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.307719 (+ 698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.310007 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.310008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.310309 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.310546 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.310548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.310548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.310624 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.310892 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.310892 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.310928 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.311046 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.311201 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":59,"thread_start_us":139,"threads_started":1}
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.126@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
I20260809 00:37:53.322275 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.315039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.7.105.125:38399)
0809 00:37:53.315077 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.315079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.315239 (+ 160us) server_negotiation.cc:407] Connection header received
0809 00:37:53.315337 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.315338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.315349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.315398 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.315820 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.316333 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.317180 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.317363 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.317442 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.319713 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.319714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.319715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.319722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.321052 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.321463 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.321465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.321465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.321521 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.321731 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.321732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.321733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.321840 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.321910 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.322099 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.322178 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:37:53.322376 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.314899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37470)
0809 00:37:53.315123 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.315174 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.315267 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.315416 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.315421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.315691 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.315741 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.315742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.316424 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.316426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.317086 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.317088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.317442 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.317443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.317525 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.318005 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.318009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.319600 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.321153 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.321161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.321164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.321376 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.321592 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.321593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.321594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.321656 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.321914 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.321923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.321976 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.322074 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.322246 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
I20260809 00:37:53.323181 14486 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.105.125:38399: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
member_type: NON_VOTER
last_known_addr {
host: "127.7.105.125"
port: 38399
}
attrs {
promote: true
}
}
dest_uuid: "0a5aa01c47434b4586800d33a846ca77"
cas_config_opid_index: -1
I20260809 00:37:53.323524 14486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } }
I20260809 00:37:53.324138 15131 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.324466 15074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:53.325299 15074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 00:37:53.325680 15129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:53.326319 15131 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.105.125:38399
I20260809 00:37:53.326478 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.326555 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:53.326556 15025 master.cc:537] Master@127.7.105.125:38399 shutting down...
I20260809 00:37:53.327793 15074 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.329643 15138 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.329727 15138 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.329808 15109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5aa01c47434b4586800d33a846ca77. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.329900 15109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.329995 15137 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.330058 15137 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.330220 15133 mvcc.cc:205] Tried to move back new op lower bound from 7316422134060642304 to 7316422133919719424. Current Snapshot: MvccSnapshot[applied={T|T < 7316422133931212800}]
I20260809 00:37:53.330420 15025 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Raft consensus shutting down.
I20260809 00:37:53.330507 15025 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Raft consensus is shut down!
I20260809 00:37:53.330541 15025 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: stopping tablet replica
I20260809 00:37:53.344139 15025 master.cc:552] Master@127.7.105.125:38399 shutdown complete.
I20260809 00:37:53.344204 15025 master_runner.cc:305] Clearing existing system tablet
I20260809 00:37:53.344344 15025 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.344393 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.345202 15025 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:53.345803 15143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.345968 15025 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.346031 15025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:53.346057 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "443e9684f110b7020caedb049f66017c"
server_key_iv: "5daea1eaa7b77fad2035a5155fb4720c"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.346853 15025 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:53.348768 15025 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:37:53.348848 15025 log.cc:1196] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 00:37:53.349093 15025 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Deleting consensus metadata
I20260809 00:37:53.349222 15025 master_runner.cc:315] Copying system tablet from 127.7.105.126:43213
I20260809 00:37:53.349881 15025 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: tablet copy: Beginning tablet copy session from remote peer at address 127.7.105.126:43213
I20260809 00:37:53.356645 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.350315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.7.105.126:43213)
0809 00:37:53.350368 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.350370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.350454 (+ 84us) server_negotiation.cc:407] Connection header received
0809 00:37:53.350522 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.350522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.350532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.350586 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.350983 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.351530 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.352678 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.352869 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.352942 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.353992 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.353994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.353994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.354001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.355433 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.355820 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.355821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.355821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.355875 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.356134 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.356135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.356136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.356265 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.356338 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.356507 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.356577 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:37:53.356704 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.350213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58356)
0809 00:37:53.350365 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.350377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.350454 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.350621 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.350622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.350830 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.350901 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.350903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.351616 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.351618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.352563 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.352566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.352949 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.352951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.353020 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.353492 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.353497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.353901 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.355519 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.355520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.355520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.355740 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.355943 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.355944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.355944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.356020 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.356349 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.356350 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.356399 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.356474 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.356592 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260809 00:37:53.357048 14511 tablet_copy_service.cc:140] P 0a5aa01c47434b4586800d33a846ca77: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff0a616e88fa4dc69d8bce30e7d1f82c ({username='oryx', principal='oryx/127.7.105.125@KRBTEST.COM'} at 127.0.0.1:58356)
I20260809 00:37:53.357156 14511 tablet_copy_service.cc:161] P 0a5aa01c47434b4586800d33a846ca77: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff0a616e88fa4dc69d8bce30e7d1f82c at {username='oryx', principal='oryx/127.7.105.125@KRBTEST.COM'} at 127.0.0.1:58356: session id = ff0a616e88fa4dc69d8bce30e7d1f82c-00000000000000000000000000000000
I20260809 00:37:53.358042 14511 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:37:53.358709 15025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:37:53.359679 15025 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: tablet copy: Starting download of 0 data blocks...
I20260809 00:37:53.359850 15025 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: tablet copy: Starting download of 1 WAL segments...
I20260809 00:37:53.360847 15025 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:37:53.361706 14511 tablet_copy_service.cc:342] P 0a5aa01c47434b4586800d33a846ca77: Request end of tablet copy session ff0a616e88fa4dc69d8bce30e7d1f82c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.105.125@KRBTEST.COM'} at 127.0.0.1:58356
I20260809 00:37:53.361773 14511 tablet_copy_service.cc:433] P 0a5aa01c47434b4586800d33a846ca77: ending tablet copy session ff0a616e88fa4dc69d8bce30e7d1f82c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff0a616e88fa4dc69d8bce30e7d1f82c
I20260809 00:37:53.364645 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.366024 15152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.366056 15153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.366083 15156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.368309 15025 server_base.cc:1034] running on GCE node
I20260809 00:37:53.368455 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.368670 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.369812 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786235873369804 us; error 29 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.371428 15025 init.cc:377] Logged in from keytab as oryx/127.7.105.125@KRBTEST.COM (short username oryx)
I20260809 00:37:53.371738 15025 webserver.cc:460] Webserver started at http://127.7.105.125:36151/ using document root <none> and password file <none>
I20260809 00:37:53.371870 15025 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.371934 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.372653 15025 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.373296 15163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.373461 15025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:53.373522 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "443e9684f110b7020caedb049f66017c"
server_key_iv: "5daea1eaa7b77fad2035a5155fb4720c"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.373590 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.382861 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.384867 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.388263 15025 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.125:38399
I20260809 00:37:53.388298 15225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.125:38399 every 8 connection(s)
I20260809 00:37:53.388630 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
I20260809 00:37:53.389551 15226 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:37:53.390136 15226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Bootstrap starting.
I20260809 00:37:53.392647 15226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:37:53.392879 15226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: Bootstrap complete.
I20260809 00:37:53.393112 15226 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } }
I20260809 00:37:53.393208 15226 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff0a616e88fa4dc69d8bce30e7d1f82c, State: Initialized, Role: LEARNER
I20260809 00:37:53.393250 15226 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } }
I20260809 00:37:53.393432 15226 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:53.393455 15229 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } }
I20260809 00:37:53.393591 15229 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.395419 15241 catalog_manager.cc:1259] Loaded cluster ID: 0f35bb16ac2c4ea5b5cf1f077046c016
I20260809 00:37:53.395910 15241 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: loading cluster ID for follower catalog manager: success
I20260809 00:37:53.397128 15241 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: acquiring CA information for follower catalog manager: success
I20260809 00:37:53.397477 15241 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:37:53.445173 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.438019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.7.105.125:38399)
0809 00:37:53.438173 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.438175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.438183 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:53.438206 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.438207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.438217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.438296 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.439059 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.439849 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.440715 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.440934 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.441026 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.442306 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.442308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.442309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.442317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.443867 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.444326 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.444329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.444329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.444392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.444612 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.444613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.444614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.444737 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.444846 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.444956 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.445050 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.7.105.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.124.keytab.
Entry for principal oryx/127.7.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.124.keytab.
I20260809 00:37:53.475934 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:35339
--webserver_interface=127.7.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.124.keytab
--principal=oryx/127.7.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:53.559674 15245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:53.559912 15245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:53.559952 15245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:53.561228 15245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:53.561290 15245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:53.561317 15245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:53.561349 15245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:53.561375 15245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:53.562871 15245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.124.keytab
--principal=oryx/127.7.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:35339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:53.563148 15245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:53.563380 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.568781 15251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.568773 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.568974 15245 server_base.cc:1034] running on GCE node
W20260809 00:37:53.568900 15250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.569280 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.569648 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.570818 15245 hybrid_clock.cc:648] HybridClock initialized: now 1786235873570807 us; error 30 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.573685 15245 init.cc:377] Logged in from keytab as oryx/127.7.105.124@KRBTEST.COM (short username oryx)
I20260809 00:37:53.574193 15245 webserver.cc:460] Webserver started at http://127.7.105.124:33327/ using document root <none> and password file <none>
I20260809 00:37:53.574431 15245 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.574478 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.574657 15245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:53.575740 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/instance:
uuid: "cae79428707e4c7094364a1f6cefb8cf"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "66ad2c8d81a907d3f6b15a23be4013e4"
server_key_iv: "33275639df90c10eca003753474b7885"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.576179 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal/instance:
uuid: "cae79428707e4c7094364a1f6cefb8cf"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "66ad2c8d81a907d3f6b15a23be4013e4"
server_key_iv: "33275639df90c10eca003753474b7885"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.577628 15245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:53.578361 15261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.578550 15245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:53.578665 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "cae79428707e4c7094364a1f6cefb8cf"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "66ad2c8d81a907d3f6b15a23be4013e4"
server_key_iv: "33275639df90c10eca003753474b7885"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.578738 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.585968 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.588317 15245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:53.588486 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.593115 15245 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.124:35339
I20260809 00:37:53.593146 15320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.124:35339 every 8 connection(s)
I20260809 00:37:53.593547 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
I20260809 00:37:53.594796 15321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:53.596618 15321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:53.601608 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 15245
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.124@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:53.605732 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.597392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.7.105.126:43213)
0809 00:37:53.597473 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.597475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.597619 (+ 144us) server_negotiation.cc:407] Connection header received
0809 00:37:53.598066 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.598067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.598078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.598141 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.598747 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.599248 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.600281 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.600486 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.600573 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.602772 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.602774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.602774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.602784 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.604227 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.604640 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.604641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.604642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.604703 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.604953 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.604954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.604955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.605124 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.605193 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.605519 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.605611 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:37:53.605870 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.597305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58362)
0809 00:37:53.597483 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.597526 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.597981 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.598214 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.598217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.598458 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.598671 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.598675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.599326 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.599327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.600193 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.600196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.600610 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.600611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.600749 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.601229 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.601237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.602672 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.604321 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.604328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.604330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.604561 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.604771 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.604772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.604773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.604883 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.605185 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.605192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.605332 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.605453 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.605639 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260809 00:37:53.606385 15321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.124@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
I20260809 00:37:53.611435 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.602252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.7.105.124:35339)
0809 00:37:53.602499 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.602503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.602518 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:37:53.602535 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.602537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.602558 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.602642 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.603315 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.604242 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.604978 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.605312 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.605402 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.608219 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.608226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.608226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.608232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.610036 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.610474 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.610477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.610477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.610581 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.610790 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.610791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.610792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.610939 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.611083 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.611240 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.611314 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260809 00:37:53.613700 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.606682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37506)
0809 00:37:53.606731 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.606742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.606780 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.606970 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.606970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.607104 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.607144 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.607145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.607825 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.607825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.608502 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.608505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.608930 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.608931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.609012 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.609297 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.609303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.610781 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.612633 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.612634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.612635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.612803 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.613030 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.613031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.613032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.613117 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.613377 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.613378 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.613428 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.613516 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.613611 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:37:53.613730 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.606783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.7.105.125:38399)
0809 00:37:53.606847 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.606849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.606856 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:53.606882 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.606882 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.606901 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.606966 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.607223 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.607713 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.608578 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.608856 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.608916 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.610875 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.610877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.610877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.610885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.612534 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.612891 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.612892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.612893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.612960 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.613196 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.613197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.613197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.613310 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.613386 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.613542 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.613628 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:37:53.614152 15321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.124@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
I20260809 00:37:53.620728 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.614467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.7.105.124:35339)
0809 00:37:53.614523 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.614524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.614531 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:53.614549 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.614565 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.614574 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.614615 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.614795 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.615246 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.615852 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.616113 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.616181 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.617953 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.617954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.617954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.617960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.619335 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.619755 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.619756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.619756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.619827 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.620174 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.620175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.620175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.620310 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.620383 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.620512 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.620634 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":39}
I20260809 00:37:53.620790 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.614404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58132)
0809 00:37:53.614466 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.614477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.614513 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.614624 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.614625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.614729 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.614756 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.614757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.615312 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.615313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.615760 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.615761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.616190 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.616196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.616243 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.616573 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.616577 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.617864 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.619447 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.619448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.619449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.619624 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.619928 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.619929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.619930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.619991 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.620387 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.620388 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.620427 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.620483 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.620570 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:37:53.622483 15321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Bootstrap starting.
I20260809 00:37:53.623355 15321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:53.623598 15321 log.cc:824] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:53.624432 15321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: No bootstrap required, opened a new log
I20260809 00:37:53.625910 15321 raft_consensus.cc:348] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } }
I20260809 00:37:53.626036 15321 raft_consensus.cc:374] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:53.626058 15321 raft_consensus.cc:729] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae79428707e4c7094364a1f6cefb8cf, State: Initialized, Role: FOLLOWER
I20260809 00:37:53.626194 15321 consensus_queue.cc:260] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } }
I20260809 00:37:53.626570 15321 sys_catalog.cc:564] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:53.626545 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } } }
I20260809 00:37:53.626673 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:37:53.628474 15338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:53.628532 15338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:53.631116 15245 master_runner.cc:428] Detected that this master cae79428707e4c7094364a1f6cefb8cf is joining an existing cluster
I20260809 00:37:53.631170 15245 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.105.124:35339
I20260809 00:37:53.660543 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.7.105.124:35339)
0809 00:37:53.632400 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.632403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.632565 (+ 162us) server_negotiation.cc:407] Connection header received
0809 00:37:53.632693 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.632694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.632704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.632735 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.633174 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.633924 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635002 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.635281 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635323 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.657047 (+ 21724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.657049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.657049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.657057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.659180 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659674 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.659675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.659675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.659728 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659912 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.659912 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.659913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.660096 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.660165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.660346 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.660433 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:37:53.660871 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.7.105.125:38399)
0809 00:37:53.632859 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.632861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.633110 (+ 249us) server_negotiation.cc:407] Connection header received
0809 00:37:53.633222 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.633223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.633234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.633264 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.633725 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.634472 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635445 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.635735 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635776 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.657129 (+ 21353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.657132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.657132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.657142 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.659239 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659756 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.659757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.659758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.659822 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.660274 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.660275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.660275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.660415 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.660466 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.660679 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.660773 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:37:53.660934 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58142)
0809 00:37:53.632481 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.632494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.632602 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.632799 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.632800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.633012 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.633091 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.633093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.634015 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.634016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.634901 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.634904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.635395 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.635461 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.635972 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.635977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.656898 (+ 20921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.659314 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.659315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.659593 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.659797 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.659798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.659848 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.660156 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.660158 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.660227 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.660733 (+ 506us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.660835 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260809 00:37:53.661212 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37512)
0809 00:37:53.633047 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.633066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.633147 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.633330 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.633331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.633557 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.633641 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.633642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.634554 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.634554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.635345 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.635347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.635859 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.635929 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.636357 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.636361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.656948 (+ 20587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.659352 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.659354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.659652 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.659894 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.659895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.659895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.660201 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.660515 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.660516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.660556 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.660993 (+ 437us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.661102 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":62,"threads_started":1}
I20260809 00:37:53.662130 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58372)
0809 00:37:53.632871 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.632880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.632959 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.633149 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.633150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.633381 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.633451 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.633453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.634416 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.634416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.635345 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.635347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.636288 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.636289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.636357 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.636710 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.636713 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.657448 (+ 20735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.660158 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.660158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.660159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.660377 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.661431 (+ 1054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.661431 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.661432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.661501 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.661809 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.661809 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.661840 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.661917 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.662024 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":82,"threads_started":1}
I20260809 00:37:53.662125 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.632631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.7.105.126:43213)
0809 00:37:53.632674 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.632676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.632959 (+ 283us) server_negotiation.cc:407] Connection header received
0809 00:37:53.633018 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.633019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.633032 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.633083 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.633540 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.634323 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.635445 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.636153 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.636188 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.657564 (+ 21376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.657567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.657567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.657577 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.660017 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.660468 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.660470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.660470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.660527 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.661605 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.661607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.661607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.661735 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.661770 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.661947 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.662024 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:37:53.662694 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14561
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235872, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.126@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
I20260809 00:37:53.673619 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:35213
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:43213
--webserver_port=43107
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:53.677523 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.668959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.7.105.124:35339)
0809 00:37:53.668998 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.669000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.669027 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:37:53.669116 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.669117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.669127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.669176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.669484 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.669956 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.670677 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.670880 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.670967 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.672865 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.672867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.672868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.672875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.676311 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.676717 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.676719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.676719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.676778 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.676990 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.676991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.676992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.677114 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.677190 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.677345 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.677426 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:53.677524 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.668916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58154)
0809 00:37:53.668982 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.668993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.669044 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.669178 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.669179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.669348 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.669394 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.669396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.670039 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.670041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.670570 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.670572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.670969 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.670970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.671041 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.671466 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.671472 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.672777 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.676410 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.676412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.676412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.676627 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.676858 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.676859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.676860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.676923 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.677183 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.677184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.677241 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.677319 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.677428 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:37:53.678450 14486 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.105.124:35339: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf"
member_type: NON_VOTER
last_known_addr {
host: "127.7.105.124"
port: 35339
}
attrs {
promote: true
}
}
dest_uuid: "0a5aa01c47434b4586800d33a846ca77"
cas_config_opid_index: 5
I20260809 00:37:53.678746 14486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } }
I20260809 00:37:53.679278 15129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cae79428707e4c7094364a1f6cefb8cf (127.7.105.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.680290 15129 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.105.124:35339
I20260809 00:37:53.680366 15129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.680435 15129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:53.680728 15245 master.cc:537] Master@127.7.105.124:35339 shutting down...
I20260809 00:37:53.680866 15293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:53.682024 15293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:37:53.682550 15131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:37:53.683568 14468 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 00:37:53.684353 15245 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 00:37:53.684373 14468 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 -> Peer cae79428707e4c7094364a1f6cefb8cf (127.7.105.124:35339): Couldn't send request to peer cae79428707e4c7094364a1f6cefb8cf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:37:53.684450 15245 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:53.684500 15245 tablet_replica.cc:330] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: stopping tablet replica
I20260809 00:37:53.687722 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.679232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37528)
0809 00:37:53.683810 (+ 4578us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.683822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.683875 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.684101 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.684102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.684104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:53.684147 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.684148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.685336 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.685338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.686527 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.686530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.687467 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.687468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.687540 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.687564 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.687618 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.687658 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4560}
I20260809 00:37:53.688028 15245 master.cc:552] Master@127.7.105.124:35339 shutdown complete.
I20260809 00:37:53.688138 15245 master_runner.cc:305] Clearing existing system tablet
I20260809 00:37:53.688180 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.679289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.7.105.125:38399)
0809 00:37:53.679332 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.679334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.683868 (+ 4534us) server_negotiation.cc:407] Connection header received
0809 00:37:53.683932 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.683934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.683962 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.684024 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:53.684587 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.685245 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.686622 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.687283 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.687359 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.687430 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.687877 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.687923 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:53.688427 15194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:37:53.688690 15131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 00:37:53.688966 15245 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.689122 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.689165 15194 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cae79428707e4c7094364a1f6cefb8cf (127.7.105.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.689936 15245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:53.690373 15229 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5aa01c47434b4586800d33a846ca77. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.690459 15229 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.690474 15131 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: attempting to promote NON_VOTER ff0a616e88fa4dc69d8bce30e7d1f82c to VOTER
I20260809 00:37:53.690496 15229 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: NON_VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: true } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.690537 15229 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.690685 15131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } }
I20260809 00:37:53.691205 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.691095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58158)
0809 00:37:53.691134 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.691175 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:35339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 00:37:53.691468 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.691397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58172)
0809 00:37:53.691427 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.691447 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:35339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 00:37:53.691457 15194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 LEARNER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260809 00:37:53.691526 14468 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 -> Peer cae79428707e4c7094364a1f6cefb8cf (127.7.105.124:35339): Couldn't send request to peer cae79428707e4c7094364a1f6cefb8cf. Status: Network error: Client connection negotiation failed: client connection to 127.7.105.124:35339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 00:37:53.691639 15129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 00:37:53.692010 15354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.692255 15129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.692481 15194 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.693249 15129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.693327 15129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:53.693313 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.693465 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:53.693768 15353 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.693869 15353 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:53.693988 15245 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.694044 15245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260809 00:37:53.694069 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "cae79428707e4c7094364a1f6cefb8cf"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "66ad2c8d81a907d3f6b15a23be4013e4"
server_key_iv: "33275639df90c10eca003753474b7885"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.694873 15245 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:53.696195 15245 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:37:53.696275 15245 log.cc:1196] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 00:37:53.696507 15245 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Deleting consensus metadata
I20260809 00:37:53.696632 15245 master_runner.cc:315] Copying system tablet from 127.7.105.126:43213
I20260809 00:37:53.697235 15245 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: tablet copy: Beginning tablet copy session from remote peer at address 127.7.105.126:43213
I20260809 00:37:53.703722 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.697555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.7.105.126:43213)
0809 00:37:53.697612 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.697614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.697727 (+ 113us) server_negotiation.cc:407] Connection header received
0809 00:37:53.697801 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.697802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.697813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.697857 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.698273 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.698915 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.699635 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.699881 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.699951 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.701115 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.701117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.701118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.701125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.702575 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.702943 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.702944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.702945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.702999 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.703216 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.703217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.703218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.703326 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.703400 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.703552 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.703637 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:37:53.703733 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.697518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58382)
0809 00:37:53.697653 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.697662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.697743 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.697871 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.697871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.698058 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.698183 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.698186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.698994 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.698995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.699533 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.699535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.699951 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.699952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.700022 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.700681 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.700687 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.700987 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.702676 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.702678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.702679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.702855 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.703076 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.703077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.703077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.703147 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.703392 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.703393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.703438 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.703526 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.703631 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 00:37:53.704250 14511 tablet_copy_service.cc:140] P 0a5aa01c47434b4586800d33a846ca77: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cae79428707e4c7094364a1f6cefb8cf ({username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:58382)
I20260809 00:37:53.704322 14511 tablet_copy_service.cc:161] P 0a5aa01c47434b4586800d33a846ca77: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cae79428707e4c7094364a1f6cefb8cf at {username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:58382: session id = cae79428707e4c7094364a1f6cefb8cf-00000000000000000000000000000000
I20260809 00:37:53.704762 14511 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:37:53.705240 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:37:53.706393 15245 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: tablet copy: Starting download of 0 data blocks...
I20260809 00:37:53.706555 15245 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: tablet copy: Starting download of 1 WAL segments...
I20260809 00:37:53.707310 15245 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:37:53.708016 14511 tablet_copy_service.cc:342] P 0a5aa01c47434b4586800d33a846ca77: Request end of tablet copy session cae79428707e4c7094364a1f6cefb8cf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:58382
I20260809 00:37:53.708118 14511 tablet_copy_service.cc:433] P 0a5aa01c47434b4586800d33a846ca77: ending tablet copy session cae79428707e4c7094364a1f6cefb8cf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cae79428707e4c7094364a1f6cefb8cf
I20260809 00:37:53.710340 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.711637 15364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.717144 15365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.717410 15367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.720252 15245 server_base.cc:1034] running on GCE node
I20260809 00:37:53.720393 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.720588 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.721727 15245 hybrid_clock.cc:648] HybridClock initialized: now 1786235873721721 us; error 34 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.723214 15245 init.cc:377] Logged in from keytab as oryx/127.7.105.124@KRBTEST.COM (short username oryx)
I20260809 00:37:53.723523 15245 webserver.cc:460] Webserver started at http://127.7.105.124:36251/ using document root <none> and password file <none>
I20260809 00:37:53.723649 15245 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.723686 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.724470 15245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:53.725049 15375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.725207 15245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:37:53.725258 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "cae79428707e4c7094364a1f6cefb8cf"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "66ad2c8d81a907d3f6b15a23be4013e4"
server_key_iv: "33275639df90c10eca003753474b7885"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.725325 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.730525 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.732370 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.735553 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.124:35339 every 8 connection(s)
I20260809 00:37:53.735535 15245 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.124:35339
I20260809 00:37:53.735963 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
I20260809 00:37:53.739218 15438 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:37:53.739784 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Bootstrap starting.
I20260809 00:37:53.742342 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:37:53.742580 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Bootstrap complete.
I20260809 00:37:53.742794 15438 raft_consensus.cc:348] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } }
I20260809 00:37:53.742887 15438 raft_consensus.cc:729] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cae79428707e4c7094364a1f6cefb8cf, State: Initialized, Role: LEARNER
I20260809 00:37:53.742933 15438 consensus_queue.cc:260] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } }
I20260809 00:37:53.743095 15438 sys_catalog.cc:564] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:53.743080 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:53.743294 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:53.745438 15454 catalog_manager.cc:1259] Loaded cluster ID: 0f35bb16ac2c4ea5b5cf1f077046c016
I20260809 00:37:53.745502 15454 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: loading cluster ID for follower catalog manager: success
I20260809 00:37:53.746238 15454 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: acquiring CA information for follower catalog manager: success
I20260809 00:37:53.746510 15454 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:37:53.796963 15349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:53.797173 15349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:53.797196 15349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:53.798362 15349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:53.798413 15349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:53.798440 15349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:53.800007 15349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:35213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=43107
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:53.800295 15349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:53.800560 15349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.807128 15460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.807140 15459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.807394 15462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.807530 15349 server_base.cc:1034] running on GCE node
I20260809 00:37:53.807766 15349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.808184 15349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.809361 15349 hybrid_clock.cc:648] HybridClock initialized: now 1786235873809347 us; error 34 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.812220 15349 init.cc:377] Logged in from keytab as oryx/127.7.105.65@KRBTEST.COM (short username oryx)
I20260809 00:37:53.812752 15349 webserver.cc:460] Webserver started at http://127.7.105.65:43107/ using document root <none> and password file <none>
I20260809 00:37:53.813021 15349 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.813117 15349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.814558 15349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:53.815229 15471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.815407 15349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:53.815492 15349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "217efa1ae2894d0592fa498828bbd9d6"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "65ee0ebc9b806139d872954bd9fc22e2"
server_key_iv: "af2de1fcfab657b86b28baf214a9933e"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.815848 15349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.828346 15349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.830955 15349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:53.831132 15349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.831404 15349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:53.831744 15349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:53.831780 15349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.831811 15349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:53.831859 15349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.838642 15349 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35213
I20260809 00:37:53.838709 15608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35213 every 8 connection(s)
I20260809 00:37:53.839155 15349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:53.840350 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 15349
I20260809 00:37:53.840535 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14721
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
I20260809 00:37:53.853199 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.840428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:34521 (local address 127.7.105.125:38399)
0809 00:37:53.840486 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.840488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.842273 (+ 1785us) server_negotiation.cc:407] Connection header received
0809 00:37:53.842377 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.842379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.842391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.842423 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.843374 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.844486 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.846047 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.846315 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.846354 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.849585 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.849587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.849588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.849597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.851336 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.851876 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.851878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.851878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.851956 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.852302 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.852304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.852304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.852475 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.852525 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.852954 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.853050 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:37:53.853637 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:35505
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:43213
--webserver_port=34455
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:53.853786 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.840349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.7.105.65:34521)
0809 00:37:53.842214 (+ 1865us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.842223 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.842298 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.842490 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.842491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.842889 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.843247 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.843257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.844592 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.844593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.845952 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.845955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.846436 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.846437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.846577 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.847286 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.847298 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.849458 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.851427 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.851435 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.851438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.851790 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.852038 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.852039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.852039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.852211 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.852595 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.852604 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.852787 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.853300 (+ 513us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.853503 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"thread_start_us":65,"threads_started":1}
I20260809 00:37:53.855751 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.839950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:42335 (local address 127.7.105.126:43213)
0809 00:37:53.840013 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.840015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.840700 (+ 685us) server_negotiation.cc:407] Connection header received
0809 00:37:53.841984 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.841986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.842003 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.842074 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.847084 (+ 5010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.847803 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.848794 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.849074 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.849139 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.852684 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.852686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.852686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.852695 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.854363 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.854779 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.854781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.854781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.854853 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.855121 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.855122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.855122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.855267 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.855327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.855546 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.855631 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:37:53.855835 15612 heartbeater.cc:347] Connected to a master server at 127.7.105.125:38399
I20260809 00:37:53.855948 15612 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.856201 15612 heartbeater.cc:511] Master 127.7.105.125:38399 requested a full tablet report, sending...
I20260809 00:37:53.856488 15616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.839969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:42335)
0809 00:37:53.840563 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.840613 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.841852 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.842243 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.842250 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.845141 (+ 2891us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.845239 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.845241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.847908 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.847909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.848693 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.848695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.849199 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.849200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.849263 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.849759 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.849763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.852573 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.854470 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.854471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.854472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.854684 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.854965 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.854966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.854966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.855031 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.855368 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.855370 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.855412 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.856155 (+ 743us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.856338 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":136,"threads_started":1}
I20260809 00:37:53.856799 15182 ts_manager.cc:194] Registered new tserver with Master: 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
I20260809 00:37:53.857133 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.839968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.7.105.65:51369)
0809 00:37:53.842031 (+ 2063us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.842043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.842102 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.842284 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.842285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.845315 (+ 3030us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.845373 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.845374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.848841 (+ 3467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.848842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.849759 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.849762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.850345 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.850347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.850416 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.850823 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.850829 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.852591 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.855930 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.855931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.855932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.856169 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.856449 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.856450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.856451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.856514 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.856818 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.856819 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.856865 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.856921 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.857021 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2029,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
I20260809 00:37:53.857137 15611 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:53.857205 15611 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.857270 15611 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
I20260809 00:37:53.857501 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.839945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:51369 (local address 127.7.105.124:35339)
0809 00:37:53.840112 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.840115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.842099 (+ 1984us) server_negotiation.cc:407] Connection header received
0809 00:37:53.842159 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.842160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.842175 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.842587 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.845448 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.846954 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.849871 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.850221 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.850287 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.852682 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.852684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.852684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.852692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.855830 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.856253 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.856254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.856255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.856322 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.856592 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.856593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.856594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.856730 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.856789 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.857269 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.857375 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":62,"threads_started":1}
I20260809 00:37:53.857532 14486 ts_manager.cc:194] Re-registered known tserver with Master: 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
I20260809 00:37:53.857916 15609 heartbeater.cc:347] Connected to a master server at 127.7.105.124:35339
I20260809 00:37:53.857976 15609 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.858052 15609 heartbeater.cc:511] Master 127.7.105.124:35339 requested a full tablet report, sending...
I20260809 00:37:53.858045 14486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:42335
I20260809 00:37:53.858454 15394 ts_manager.cc:194] Registered new tserver with Master: 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
W20260809 00:37:53.942199 15620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:53.942416 15620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:53.942442 15620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:53.943882 15620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:53.943944 15620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:53.943990 15620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:53.945677 15620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:35505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=34455
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:53.945957 15620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:53.946208 15620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:53.951525 15630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.951548 15626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:53.951552 15627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:53.951661 15620 server_base.cc:1034] running on GCE node
I20260809 00:37:53.951973 15620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:53.952246 15620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:53.953433 15620 hybrid_clock.cc:648] HybridClock initialized: now 1786235873953408 us; error 35 us; skew 500 ppm
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:53.956236 15620 init.cc:377] Logged in from keytab as oryx/127.7.105.66@KRBTEST.COM (short username oryx)
I20260809 00:37:53.956681 15620 webserver.cc:460] Webserver started at http://127.7.105.66:34455/ using document root <none> and password file <none>
I20260809 00:37:53.956897 15620 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:53.956959 15620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:53.958294 15620 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:53.959035 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.959270 15620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:53.959367 15620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "942f1a77b5f748eb997b6acd4ed2308b"
format_stamp: "Formatted at 2026-08-09 00:37:52 on dist-test-slave-1zqn"
server_key: "42900b91e6e7387c665196b304d8c636"
server_key_iv: "aa4d65cf2efd7c6aea6f1c059e80be61"
server_key_version: "encryptionkey@0"
I20260809 00:37:53.959694 15620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:53.967971 15620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:53.970325 15620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:53.970439 15620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:53.970611 15620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:53.970923 15620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:53.970966 15620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.970990 15620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:53.971038 15620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:53.977957 15620 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35505
I20260809 00:37:53.978016 15764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35505 every 8 connection(s)
I20260809 00:37:53.978390 15620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:53.981429 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 15620
I20260809 00:37:53.981607 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14870
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
Aug 09 00:37:53 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
I20260809 00:37:53.992358 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:53011 (local address 127.7.105.124:35339)
0809 00:37:53.979326 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.979329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.979490 (+ 161us) server_negotiation.cc:407] Connection header received
0809 00:37:53.980664 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.980665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.980680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.980748 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.981715 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.982438 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984218 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984635 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984709 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.988574 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.988576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.988576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.988585 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.990611 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.991136 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.991137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.991138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.991200 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.991490 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.991492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.991492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.991634 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.991699 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.992133 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.992231 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 00:37:53.992972 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.7.105.66:53011)
0809 00:37:53.979420 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.979431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.980553 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.980829 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.980831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.981288 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.981597 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.981604 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.982554 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.982555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984108 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.984112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984777 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984862 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.985379 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.985392 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.988467 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.990711 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.990719 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.990722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.991043 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.991272 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.991273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.991273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.991415 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.991728 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.991735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.991912 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.992037 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.992685 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":25,"thread_start_us":104,"threads_started":1}
I20260809 00:37:53.993191 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:49549 (local address 127.7.105.125:38399)
0809 00:37:53.979614 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.979616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.979708 (+ 92us) server_negotiation.cc:407] Connection header received
0809 00:37:53.980684 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.980685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.980698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.980732 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.981713 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.982646 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984176 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984495 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984534 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.987988 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.987990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.987990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.988000 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.991914 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.992335 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.992336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.992336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.992396 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.992627 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.992628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.992628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.992757 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.992811 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.992996 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.993085 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:37:53.993562 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.7.105.66:49549)
0809 00:37:53.979632 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.979641 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.980566 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.980826 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.980831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.981272 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.981596 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.981603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.983049 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.983050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984080 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.984082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984621 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984758 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.985647 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.985653 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.987867 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.992007 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.992008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.992008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.992247 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.992491 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.992492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.992492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.992553 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.992845 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.992845 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.992884 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.993301 (+ 417us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.993416 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":31,"thread_start_us":73,"threads_started":1}
I20260809 00:37:53.994634 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:50539 (local address 127.7.105.126:43213)
0809 00:37:53.979478 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:53.979480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:53.979493 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:37:53.980683 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:53.980685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:53.980698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:53.980732 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:53.982734 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.983475 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984440 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984719 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984758 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.989617 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:53.989618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:53.989619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:53.989628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:53.991337 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.993780 (+ 2443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.993782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.993782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.993849 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:53.994092 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:53.994093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:53.994093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:53.994213 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:53.994286 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:53.994464 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:37:53.994544 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:37:53.994817 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:53.979112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:50539)
0809 00:37:53.979295 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:53.979337 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:53.980554 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:53.980826 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:53.980831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:53.981263 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:53.981596 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.981603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.983570 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.983571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984353 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:53.984354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:53.984835 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:53.984836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:53.984915 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:53.985534 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:53.985539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:53.989512 (+ 3973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:53.993489 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.993490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.993490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.993710 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.993948 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:53.993949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:53.993949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:53.994016 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:53.994289 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:53.994291 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:53.994332 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 00:37:53.994416 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:37:53.994515 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":7,"thread_start_us":99,"threads_started":1}
I20260809 00:37:53.997251 15767 heartbeater.cc:347] Connected to a master server at 127.7.105.125:38399
I20260809 00:37:53.997363 15767 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.997399 15766 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:53.997464 15766 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.997561 15766 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
I20260809 00:37:53.997565 15767 heartbeater.cc:511] Master 127.7.105.125:38399 requested a full tablet report, sending...
I20260809 00:37:53.998054 14486 ts_manager.cc:194] Re-registered known tserver with Master: 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505)
I20260809 00:37:53.998138 15182 ts_manager.cc:194] Registered new tserver with Master: 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505)
I20260809 00:37:53.998566 14486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:50539
I20260809 00:37:53.999707 15765 heartbeater.cc:347] Connected to a master server at 127.7.105.124:35339
I20260809 00:37:53.999783 15765 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:53.999854 15765 heartbeater.cc:511] Master 127.7.105.124:35339 requested a full tablet report, sending...
I20260809 00:37:54.000113 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:38267
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:43213
--webserver_port=41495
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:54.000264 15394 ts_manager.cc:194] Registered new tserver with Master: 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505)
W20260809 00:37:54.084795 15773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:54.085085 15773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:54.085115 15773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:54.086302 15773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:54.086350 15773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:54.086377 15773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:54.087905 15773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:38267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=41495
--tserver_master_addrs=127.7.105.126:43213,127.7.105.125:38399,127.7.105.124:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:54.088195 15773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:54.088454 15773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:54.093837 15779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:54.093837 15778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:54.094167 15773 server_base.cc:1034] running on GCE node
W20260809 00:37:54.093879 15782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:54.094492 15773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:54.094756 15773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:54.095922 15773 hybrid_clock.cc:648] HybridClock initialized: now 1786235874095935 us; error 82 us; skew 500 ppm
Aug 09 00:37:54 dist-test-slave-1zqn krb5kdc[14435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:54.098698 15773 init.cc:377] Logged in from keytab as oryx/127.7.105.67@KRBTEST.COM (short username oryx)
I20260809 00:37:54.099249 15773 webserver.cc:460] Webserver started at http://127.7.105.67:41495/ using document root <none> and password file <none>
I20260809 00:37:54.099483 15773 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:54.099567 15773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:54.101078 15773 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:54.101780 15789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:54.101939 15773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:54.102106 15773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "b4622e06bd874a13bcfe35bc50ee7087"
format_stamp: "Formatted at 2026-08-09 00:37:53 on dist-test-slave-1zqn"
server_key: "e5441d52f771cd8da24d7521ef7752f6"
server_key_iv: "dee2b99d365cf8b5f4812a2dd322b3d4"
server_key_version: "encryptionkey@0"
I20260809 00:37:54.102454 15773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:54.115168 15773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:54.117426 15773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:54.117597 15773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:54.117849 15773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:37:54.118222 15773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:54.118274 15773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:54.118319 15773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:54.118345 15773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:54.124622 15773 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38267
I20260809 00:37:54.124753 15919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38267 every 8 connection(s)
I20260809 00:37:54.125111 15773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:54.125778 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 15773
Aug 09 00:37:54 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.125@KRBTEST.COM
Aug 09 00:37:54 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.124@KRBTEST.COM
I20260809 00:37:54.138127 15934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.7.105.124:35339)
0809 00:37:54.127433 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.127435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.127609 (+ 174us) server_negotiation.cc:407] Connection header received
0809 00:37:54.127635 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.127635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.127648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.127705 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.128557 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.129359 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.130412 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.130715 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.130776 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.133086 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.133087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.133088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.133095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.135861 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.136547 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.136549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.136549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.136617 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.136833 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.136834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.136834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.136989 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.137088 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.137920 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.138010 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":59,"threads_started":1}
I20260809 00:37:54.138481 15925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.125986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.7.105.67:38151)
0809 00:37:54.126256 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.126301 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.127253 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.128100 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.128101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.128307 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:54.128445 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.128453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.129473 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.129474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.130847 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.130850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.131350 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.131351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.131516 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.132023 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:54.132035 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:54.134667 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:54.136576 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.136583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.136585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.136919 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.137125 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.137126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.137126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.137261 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.137602 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:54.137619 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:54.137786 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.137903 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.138072 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":40,"thread_start_us":171,"threads_started":1}
I20260809 00:37:54.138499 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.128437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58188)
0809 00:37:54.128520 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.128531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.128568 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.134826 (+ 6258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.134828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.134830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.134878 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.134879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.135838 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.135839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.137066 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.137070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.138271 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.138272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.138333 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.138338 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.138366 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.138412 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 00:37:54.138604 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:38151 (local address 127.7.105.125:38399)
0809 00:37:54.126102 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.126105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.126397 (+ 292us) server_negotiation.cc:407] Connection header received
0809 00:37:54.127384 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.127386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.127400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.127462 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.128571 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.129330 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.130938 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.131212 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.131282 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.134783 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.134784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.134785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.134792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.136487 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.137000 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.137001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.137001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.137055 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.137334 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.137335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.137336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.137497 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.137570 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.138333 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.138499 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:54.139457 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.128642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.7.105.124:35339)
0809 00:37:54.134666 (+ 6024us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.134668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.134675 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:54.134700 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.134701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.134744 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.134822 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.134958 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.135736 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.137497 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.138155 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.139213 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.139305 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.139326 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.139379 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5999,"thread_start_us":67,"threads_started":1}
I20260809 00:37:54.139629 15923 heartbeater.cc:347] Connected to a master server at 127.7.105.125:38399
I20260809 00:37:54.139691 15406 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 00:37:54.139747 15923 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:54.139926 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5aa01c47434b4586800d33a846ca77. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:54.140004 15923 heartbeater.cc:511] Master 127.7.105.125:38399 requested a full tablet report, sending...
I20260809 00:37:54.140038 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: LEARNER
I20260809 00:37:54.140551 15182 ts_manager.cc:194] Registered new tserver with Master: b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
Aug 09 00:37:54 dist-test-slave-1zqn krb5kdc[14435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:54.142632 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:44603 (local address 127.7.105.124:35339)
0809 00:37:54.126434 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.126437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.126497 (+ 60us) server_negotiation.cc:407] Connection header received
0809 00:37:54.129809 (+ 3312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.129810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.129820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.129849 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.130217 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.130930 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.131932 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.132287 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.132327 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.139223 (+ 6896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.139224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.139225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.139232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.141348 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141768 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.141769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.141770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.141826 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.142035 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.142036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.142036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.142167 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.142209 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.142451 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.142530 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:37:54.142937 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.7.105.67:44603)
0809 00:37:54.126452 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.126461 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.129722 (+ 3261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.129911 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.129911 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.130073 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:54.130134 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.130136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.131022 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.131023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.131834 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.131837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.132403 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.132404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.132468 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.132912 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:54.132917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:54.139100 (+ 6183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:54.141428 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.141429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.141673 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.141902 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.141903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.141961 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.142262 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:54.142263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:54.142300 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.142720 (+ 420us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.142832 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260809 00:37:54.142946 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.127004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.7.105.126:43213)
0809 00:37:54.128836 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.128838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.128844 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:54.128862 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.128863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.128873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.128929 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.129658 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.130439 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.133950 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.134260 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.134322 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.137143 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.137144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.137144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.137152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.141401 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141815 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.141816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.141816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.141877 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.142080 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.142081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.142081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.142218 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.142308 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.142763 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.142850 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1804,"thread_start_us":95,"threads_started":1}
I20260809 00:37:54.143429 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.127744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.7.105.125:38399)
0809 00:37:54.128024 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.128027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.131561 (+ 3534us) server_negotiation.cc:407] Connection header received
0809 00:37:54.131585 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.131586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.131599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.131655 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.132282 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.132993 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.133921 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.134290 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.134345 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.136024 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.136025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.136026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.136032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.139446 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141638 (+ 2192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.141639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.141639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.141701 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.141907 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.141907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.141908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.142036 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.142128 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.142540 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.142648 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":66,"threads_started":1}
I20260809 00:37:54.143455 15920 heartbeater.cc:347] Connected to a master server at 127.7.105.124:35339
I20260809 00:37:54.143497 15920 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:54.143570 15920 heartbeater.cc:511] Master 127.7.105.124:35339 requested a full tablet report, sending...
I20260809 00:37:54.143920 15394 ts_manager.cc:194] Registered new tserver with Master: b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
I20260809 00:37:54.146051 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:52837)
0809 00:37:54.127087 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.127097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.127270 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.127621 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.127626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.128110 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:54.128445 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.128453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.129467 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.129468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.138846 (+ 9378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.138849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.140443 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.140444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.140519 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.140899 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:54.140904 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:54.142557 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:54.144989 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.144990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.144991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.145168 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.145388 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:54.145389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:54.145389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:54.145446 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:54.145714 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:54.145715 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:54.145757 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.145834 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.145945 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":165,"threads_started":1}
I20260809 00:37:54.146036 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.126752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:52837 (local address 127.7.105.126:43213)
0809 00:37:54.126802 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.126804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.127193 (+ 389us) server_negotiation.cc:407] Connection header received
0809 00:37:54.127371 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.127373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.127385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.127429 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.128566 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.129346 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.138933 (+ 9587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.139237 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.139298 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.142651 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.142653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.142653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.142660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.144905 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.145255 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.145256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.145256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.145318 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.145527 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.145528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.145528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.145639 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.145711 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.145876 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.145948 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:54.146369 15921 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43213
I20260809 00:37:54.146427 15921 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:54.144914 14486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:37:54.146507 15921 heartbeater.cc:511] Master 127.7.105.126:43213 requested a full tablet report, sending...
W20260809 00:37:54.146695 14486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:54.146776 14485 ts_manager.cc:194] Re-registered known tserver with Master: b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
I20260809 00:37:54.147185 14485 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:52837
I20260809 00:37:54.147477 15356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: NON_VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: true } } }
I20260809 00:37:54.147580 15356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:54.149145 15355 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: attempting to promote NON_VOTER cae79428707e4c7094364a1f6cefb8cf to VOTER
I20260809 00:37:54.149286 15355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } }
I20260809 00:37:54.149698 15406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 LEARNER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 00:37:54.149899 15355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 00:37:54.150426 15193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Refusing update from remote peer 0a5aa01c47434b4586800d33a846ca77: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 00:37:54.150640 15356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 00:37:54.150642 15131 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cae79428707e4c7094364a1f6cefb8cf (127.7.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.150811 15406 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cae79428707e4c7094364a1f6cefb8cf (127.7.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.151026 15193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cae79428707e4c7094364a1f6cefb8cf (127.7.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.151649 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.151724 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:54.151738 15356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.151810 15356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:54.152133 15353 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.152230 15353 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:54.152336 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.152462 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:54.152657 15355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a5aa01c47434b4586800d33a846ca77" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:54.152746 15355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:54.157547 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.7.105.66:35505)
0809 00:37:54.153616 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.153622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.153633 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:54.153677 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.153679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.153698 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.153983 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.154065 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155166 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.156527 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157225 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157311 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.157397 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.157431 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.157469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":213,"threads_started":1}
I20260809 00:37:54.157536 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35505 (local address 127.0.0.1:56640)
0809 00:37:54.153413 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.153426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.153469 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.153813 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.153814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.153816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.153870 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.153872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155270 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.155271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.156446 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.156449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157308 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.157367 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.157371 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.157418 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.157459 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":7}
I20260809 00:37:54.157809 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38267 (local address 127.0.0.1:47826)
0809 00:37:54.153437 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.153449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.153529 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.154144 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.154145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.154147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.154193 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.154194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155208 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.155209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.156552 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.156554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157550 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.157614 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.157619 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.157669 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.157732 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":30,"thread_start_us":108,"threads_started":1}
I20260809 00:37:54.158201 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35213 (local address 127.0.0.1:45928)
0809 00:37:54.153611 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.153619 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.153651 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.154710 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.154711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.154713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.154753 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.154755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155749 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.155750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.156714 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.156717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157979 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.158041 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.158045 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.158085 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.158124 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":93,"threads_started":1}
I20260809 00:37:54.158298 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.7.105.67:38267)
0809 00:37:54.153996 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.154001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.154011 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:54.154052 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.154055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.154072 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.154148 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.154249 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155120 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.156640 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157366 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157439 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.157511 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.157935 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.157986 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":93,"threads_started":1}
I20260809 00:37:54.158587 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.153835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.7.105.65:35213)
0809 00:37:54.154499 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.154504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.154514 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:54.154591 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.154592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.154608 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.154689 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.154816 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.155660 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.156809 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.157512 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.157575 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.158459 (+ 884us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.158489 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.158530 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":197,"threads_started":1}
I20260809 00:37:54.158500 15693 tablet_service.cc:1467] Processing CreateTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b (DEFAULT_TABLE table=test-table [id=2eab3441e34a4ac0828ca1e08e3c582f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:54.158902 15693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8112ee568ee4bf89863c7a3e30b8e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:54.160707 15842 tablet_service.cc:1467] Processing CreateTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b (DEFAULT_TABLE table=test-table [id=2eab3441e34a4ac0828ca1e08e3c582f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:54.161089 15842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8112ee568ee4bf89863c7a3e30b8e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:54.160987 15527 tablet_service.cc:1467] Processing CreateTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b (DEFAULT_TABLE table=test-table [id=2eab3441e34a4ac0828ca1e08e3c582f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:54.161322 15527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8112ee568ee4bf89863c7a3e30b8e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:54.162088 15956 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Bootstrap starting.
I20260809 00:37:54.162770 15956 tablet_bootstrap.cc:654] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:54.163170 15956 log.cc:824] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:54.163920 15956 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: No bootstrap required, opened a new log
I20260809 00:37:54.164002 15956 ts_tablet_manager.cc:1397] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:54.164713 15958 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Bootstrap starting.
I20260809 00:37:54.165431 15958 tablet_bootstrap.cc:654] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:54.165772 15958 log.cc:824] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:54.166604 15958 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: No bootstrap required, opened a new log
I20260809 00:37:54.166677 15958 ts_tablet_manager.cc:1397] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:54.166706 15956 raft_consensus.cc:348] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.166880 15956 raft_consensus.cc:374] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:54.166920 15956 raft_consensus.cc:729] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942f1a77b5f748eb997b6acd4ed2308b, State: Initialized, Role: FOLLOWER
I20260809 00:37:54.167042 15956 consensus_queue.cc:260] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.167336 15956 ts_tablet_manager.cc:1428] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:54.168689 15959 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Bootstrap starting.
I20260809 00:37:54.168097 15766 heartbeater.cc:503] Master 127.7.105.126:43213 was elected leader, sending a full tablet report...
I20260809 00:37:54.169355 15959 tablet_bootstrap.cc:654] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:54.169263 15958 raft_consensus.cc:348] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.169420 15958 raft_consensus.cc:374] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:54.169468 15958 raft_consensus.cc:729] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217efa1ae2894d0592fa498828bbd9d6, State: Initialized, Role: FOLLOWER
I20260809 00:37:54.169615 15958 consensus_queue.cc:260] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.169704 15959 log.cc:824] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:54.169915 15958 ts_tablet_manager.cc:1428] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:54.170027 15611 heartbeater.cc:503] Master 127.7.105.126:43213 was elected leader, sending a full tablet report...
I20260809 00:37:54.171046 15959 tablet_bootstrap.cc:492] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: No bootstrap required, opened a new log
I20260809 00:37:54.171147 15959 ts_tablet_manager.cc:1397] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:37:54.172766 15959 raft_consensus.cc:348] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.172890 15959 raft_consensus.cc:374] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:54.172935 15959 raft_consensus.cc:729] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4622e06bd874a13bcfe35bc50ee7087, State: Initialized, Role: FOLLOWER
I20260809 00:37:54.173043 15959 consensus_queue.cc:260] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.173301 15959 ts_tablet_manager.cc:1428] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:37:54.173339 15921 heartbeater.cc:503] Master 127.7.105.126:43213 was elected leader, sending a full tablet report...
I20260809 00:37:54.195875 15965 raft_consensus.cc:482] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:54.196003 15965 raft_consensus.cc:504] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.196445 15965 leader_election.cc:288] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505), 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
I20260809 00:37:54.199548 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.196678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35505 (local address 127.7.105.67:43973)
0809 00:37:54.196794 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.196806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.196860 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.197025 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.197026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.197028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.197132 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.197134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.197761 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.197763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.198697 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.198700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.199255 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.199256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.199339 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.199343 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.199402 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.199435 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 00:37:54.199548 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.196857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:43973 (local address 127.7.105.66:35505)
0809 00:37:54.196903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.196905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.196913 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:54.196939 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.196940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.196947 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.197004 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.197218 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.197680 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.198828 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.199173 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.199230 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.199238 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.199427 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.199480 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:54.199625 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.196724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:58373 (local address 127.7.105.65:35213)
0809 00:37:54.196769 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.196772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.196781 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:54.196870 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.196871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.196882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.196934 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:54.197176 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.197761 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.198799 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.199223 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.199303 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.199311 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.199501 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.199543 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:54.199653 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.196653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35213 (local address 127.7.105.67:58373)
0809 00:37:54.196713 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:54.196725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:54.196769 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:54.196988 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:54.196990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:54.196991 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:54.197084 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.197086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.197853 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.197854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.198690 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:54.198693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.199315 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.199316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:54.199408 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.199412 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:54.199470 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:37:54.199507 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":92}
I20260809 00:37:54.200178 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8112ee568ee4bf89863c7a3e30b8e6b" candidate_uuid: "b4622e06bd874a13bcfe35bc50ee7087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942f1a77b5f748eb997b6acd4ed2308b" is_pre_election: true
I20260809 00:37:54.200313 15716 raft_consensus.cc:2454] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4622e06bd874a13bcfe35bc50ee7087 in term 0.
I20260809 00:37:54.200383 15552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8112ee568ee4bf89863c7a3e30b8e6b" candidate_uuid: "b4622e06bd874a13bcfe35bc50ee7087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217efa1ae2894d0592fa498828bbd9d6" is_pre_election: true
I20260809 00:37:54.200570 15552 raft_consensus.cc:2454] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4622e06bd874a13bcfe35bc50ee7087 in term 0.
I20260809 00:37:54.200616 15793 leader_election.cc:302] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 942f1a77b5f748eb997b6acd4ed2308b, b4622e06bd874a13bcfe35bc50ee7087; no voters:
I20260809 00:37:54.200750 15965 raft_consensus.cc:2781] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:54.200812 15965 raft_consensus.cc:482] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:54.200830 15965 raft_consensus.cc:3037] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:54.201577 15965 raft_consensus.cc:504] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.201727 15965 leader_election.cc:288] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [CANDIDATE]: Term 1 election: Requested vote from peers 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505), 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213)
I20260809 00:37:54.201983 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8112ee568ee4bf89863c7a3e30b8e6b" candidate_uuid: "b4622e06bd874a13bcfe35bc50ee7087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942f1a77b5f748eb997b6acd4ed2308b"
I20260809 00:37:54.202064 15716 raft_consensus.cc:3037] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:54.202864 15716 raft_consensus.cc:2454] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4622e06bd874a13bcfe35bc50ee7087 in term 1.
I20260809 00:37:54.203002 15552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8112ee568ee4bf89863c7a3e30b8e6b" candidate_uuid: "b4622e06bd874a13bcfe35bc50ee7087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217efa1ae2894d0592fa498828bbd9d6"
I20260809 00:37:54.203033 15793 leader_election.cc:302] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 942f1a77b5f748eb997b6acd4ed2308b, b4622e06bd874a13bcfe35bc50ee7087; no voters:
I20260809 00:37:54.203087 15552 raft_consensus.cc:3037] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:54.203141 15965 raft_consensus.cc:2781] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:54.203336 15965 raft_consensus.cc:686] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 1 LEADER]: Becoming Leader. State: Replica: b4622e06bd874a13bcfe35bc50ee7087, State: Running, Role: LEADER
I20260809 00:37:54.203665 15965 consensus_queue.cc:237] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } }
I20260809 00:37:54.204027 15552 raft_consensus.cc:2454] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4622e06bd874a13bcfe35bc50ee7087 in term 1.
I20260809 00:37:54.204607 14486 catalog_manager.cc:5495] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "b4622e06bd874a13bcfe35bc50ee7087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:54.213382 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.210713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.7.105.67:38267)
0809 00:37:54.210783 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.210785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.210809 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:37:54.210836 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.210837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.210852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.210902 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:54.211082 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.211543 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.212441 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.212620 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.212674 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.212885 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:54.212975 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:54.213033 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.213282 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.213308 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:37:54.215233 15552 raft_consensus.cc:1261] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Refusing update from remote peer b4622e06bd874a13bcfe35bc50ee7087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:54.215302 15716 raft_consensus.cc:1261] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Refusing update from remote peer b4622e06bd874a13bcfe35bc50ee7087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:54.215603 15966 consensus_queue.cc:1035] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:54.215729 15965 consensus_queue.cc:1035] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:54.218310 15969 mvcc.cc:205] Tried to move back new op lower bound from 7316422140783108096 to 7316422140739190784. Current Snapshot: MvccSnapshot[applied={T|T < 7316422140783108096}]
I20260809 00:37:54.218933 15971 mvcc.cc:205] Tried to move back new op lower bound from 7316422140783108096 to 7316422140739190784. Current Snapshot: MvccSnapshot[applied={T|T < 7316422140783108096}]
I20260809 00:37:54.221692 14486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:54.221804 14486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:54.223237 14486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a5aa01c47434b4586800d33a846ca77: Sending DeleteTablet for 3 replicas of tablet b8112ee568ee4bf89863c7a3e30b8e6b
I20260809 00:37:54.223614 15842 tablet_service.cc:1514] Processing DeleteTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:54 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:47826
I20260809 00:37:54.223645 15693 tablet_service.cc:1514] Processing DeleteTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:54 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:56640
I20260809 00:37:54.223762 15527 tablet_service.cc:1514] Processing DeleteTablet for tablet b8112ee568ee4bf89863c7a3e30b8e6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:54 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45928
I20260809 00:37:54.224042 15980 tablet_replica.cc:330] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: stopping tablet replica
I20260809 00:37:54.224099 15981 tablet_replica.cc:330] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: stopping tablet replica
I20260809 00:37:54.224166 15980 raft_consensus.cc:2229] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:54.224200 15981 raft_consensus.cc:2229] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:54.224252 15980 raft_consensus.cc:2258] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:54.224277 15981 raft_consensus.cc:2258] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:54.224615 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 14450
I20260809 00:37:54.224931 15980 ts_tablet_manager.cc:1905] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:54.227841 15980 ts_tablet_manager.cc:1918] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:54.227926 15980 log.cc:1196] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/b8112ee568ee4bf89863c7a3e30b8e6b
I20260809 00:37:54.228336 15980 ts_tablet_manager.cc:1939] T b8112ee568ee4bf89863c7a3e30b8e6b P b4622e06bd874a13bcfe35bc50ee7087: Deleting consensus metadata
I20260809 00:37:54.232216 15979 tablet_replica.cc:330] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: stopping tablet replica
I20260809 00:37:54.232475 15979 raft_consensus.cc:2229] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:54.232589 15981 ts_tablet_manager.cc:1905] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:54.232697 15979 raft_consensus.cc:2258] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:54.233312 15979 ts_tablet_manager.cc:1905] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:54.234760 15981 ts_tablet_manager.cc:1918] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:54.234859 15981 log.cc:1196] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/b8112ee568ee4bf89863c7a3e30b8e6b
I20260809 00:37:54.234953 15979 ts_tablet_manager.cc:1918] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:54.235080 15979 log.cc:1196] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/b8112ee568ee4bf89863c7a3e30b8e6b
I20260809 00:37:54.235189 15981 ts_tablet_manager.cc:1939] T b8112ee568ee4bf89863c7a3e30b8e6b P 217efa1ae2894d0592fa498828bbd9d6: Deleting consensus metadata
I20260809 00:37:54.235433 15979 ts_tablet_manager.cc:1939] T b8112ee568ee4bf89863c7a3e30b8e6b P 942f1a77b5f748eb997b6acd4ed2308b: Deleting consensus metadata
W20260809 00:37:54.235507 15790 connection.cc:535] server connection from 127.0.0.1:47826 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 00:37:54.235674 15472 connection.cc:441] server connection from 127.0.0.1:45928 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45928 (request call id 1) could send its response
I20260809 00:37:54.240556 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.223946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.7.105.124:35339)
0809 00:37:54.223993 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.223996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.232258 (+ 8262us) server_negotiation.cc:407] Connection header received
0809 00:37:54.232411 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.232412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.232425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.232457 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.233358 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.234086 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.235184 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.235497 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.235547 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.237254 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.237257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.237257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.237266 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.239056 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.239458 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.239460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.239460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.239524 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.239741 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.239741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.239742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.239881 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.239979 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.240384 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.240458 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:54.247013 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.237695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.7.105.124:35339)
0809 00:37:54.237739 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.237741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.238120 (+ 379us) server_negotiation.cc:407] Connection header received
0809 00:37:54.238221 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.238222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.238232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.238259 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.238935 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.239693 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.241410 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.241723 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.241762 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.243339 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.243341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.243341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.243348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.245711 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.246105 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.246106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.246107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.246171 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.246421 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.246422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.246423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.246546 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.246617 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.246800 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.246887 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:54.250555 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:54.238358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.7.105.125:38399)
0809 00:37:54.239177 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:54.239179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:54.239262 (+ 83us) server_negotiation.cc:407] Connection header received
0809 00:37:54.239359 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:54.239360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:54.239374 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:54.239405 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:54.240043 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.240786 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.242912 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:54.243500 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:54.243590 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:54.245699 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:54.245701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:54.245701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:54.245708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:54.249412 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.249761 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.249763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.249763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.249827 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:54.250031 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:54.250032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:54.250032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:54.250158 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:54.250264 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:54.250362 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 00:37:54.250451 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791}
I20260809 00:37:55.219790 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.219467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:43249)
0809 00:37:55.219618 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.219724 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
W20260809 00:37:55.220119 15611 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:43213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.221165 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.220921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:35629)
0809 00:37:55.221045 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.221127 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
W20260809 00:37:55.221419 15921 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:43213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.222132 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.221803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:41771)
0809 00:37:55.222002 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.222082 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":58,"threads_started":1}
W20260809 00:37:55.222329 15766 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:43213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.723389 15993 raft_consensus.cc:482] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a5aa01c47434b4586800d33a846ca77)
I20260809 00:37:55.723531 15993 raft_consensus.cc:504] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } }
I20260809 00:37:55.723793 15993 leader_election.cc:288] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213), ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125:38399)
I20260809 00:37:55.724277 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.723991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58852)
0809 00:37:55.724147 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.724217 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":113,"threads_started":1}
W20260809 00:37:55.724493 15379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.724546 15996 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a5aa01c47434b4586800d33a846ca77)
I20260809 00:37:55.724684 15996 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } }
I20260809 00:37:55.725119 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.724907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58862)
0809 00:37:55.725047 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.725084 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 00:37:55.725227 15379 leader_election.cc:334] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213): Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.725405 16000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.725034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58874)
0809 00:37:55.725164 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.725357 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260809 00:37:55.725487 15996 leader_election.cc:288] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213), cae79428707e4c7094364a1f6cefb8cf (127.7.105.124:35339)
W20260809 00:37:55.725596 15168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.726640 16000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.726540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58886)
0809 00:37:55.726598 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.726619 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 00:37:55.726732 15168 leader_election.cc:334] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213): Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.728291 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.724245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:38399 (local address 127.0.0.1:37554)
0809 00:37:55.724351 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.724363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.724451 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.724771 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.724773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.724775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.724843 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.724847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.725937 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.725938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.727274 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.727276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.728003 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.728004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.728119 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.728124 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.728194 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.728235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":45,"threads_started":1}
I20260809 00:37:55.728318 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.724431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.7.105.125:38399)
0809 00:37:55.724594 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.724597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.724608 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:55.724644 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.724645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.724659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.724760 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.724944 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.725855 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.727360 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.727898 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.727979 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.727991 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.728209 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.728256 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 00:37:55.728583 15193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae79428707e4c7094364a1f6cefb8cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" is_pre_election: true
I20260809 00:37:55.728708 15193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae79428707e4c7094364a1f6cefb8cf in term 1.
I20260809 00:37:55.728884 15379 leader_election.cc:302] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cae79428707e4c7094364a1f6cefb8cf, ff0a616e88fa4dc69d8bce30e7d1f82c; no voters: 0a5aa01c47434b4586800d33a846ca77
I20260809 00:37:55.729008 15993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:37:55.729060 15993 raft_consensus.cc:482] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a5aa01c47434b4586800d33a846ca77)
I20260809 00:37:55.729099 15993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:37:55.729353 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.725156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:35339 (local address 127.0.0.1:58222)
0809 00:37:55.725465 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.725601 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.725780 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.725980 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.725981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.725983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.726047 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.726048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.727123 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.727124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.728346 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.728349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.729157 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.729158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.729226 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.729230 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.729274 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.729307 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":50,"threads_started":1}
I20260809 00:37:55.729498 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.725379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.7.105.124:35339)
0809 00:37:55.725595 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.725597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.725699 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:37:55.725880 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.725881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.725892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.725975 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.726110 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.727025 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.728450 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.729009 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.729114 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.729127 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.729306 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.729375 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
I20260809 00:37:55.729684 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cae79428707e4c7094364a1f6cefb8cf" is_pre_election: true
I20260809 00:37:55.730235 15993 raft_consensus.cc:504] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } }
I20260809 00:37:55.730368 15406 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff0a616e88fa4dc69d8bce30e7d1f82c in current term 2: Already voted for candidate cae79428707e4c7094364a1f6cefb8cf in this term.
I20260809 00:37:55.730408 15993 leader_election.cc:288] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 election: Requested vote from peers 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213), ff0a616e88fa4dc69d8bce30e7d1f82c (127.7.105.125:38399)
I20260809 00:37:55.730589 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.730491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58898)
0809 00:37:55.730526 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.730559 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:37:55.730636 15164 leader_election.cc:302] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff0a616e88fa4dc69d8bce30e7d1f82c; no voters: 0a5aa01c47434b4586800d33a846ca77, cae79428707e4c7094364a1f6cefb8cf
I20260809 00:37:55.730631 15193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae79428707e4c7094364a1f6cefb8cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c"
I20260809 00:37:55.730715 15193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:37:55.731451 15193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae79428707e4c7094364a1f6cefb8cf in term 2.
I20260809 00:37:55.731534 15996 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:37:55.731666 15379 leader_election.cc:302] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae79428707e4c7094364a1f6cefb8cf, ff0a616e88fa4dc69d8bce30e7d1f82c; no voters:
I20260809 00:37:55.731863 15993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:37:55.732023 15993 raft_consensus.cc:686] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [term 2 LEADER]: Becoming Leader. State: Replica: cae79428707e4c7094364a1f6cefb8cf, State: Running, Role: LEADER
I20260809 00:37:55.732139 15993 consensus_queue.cc:237] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } }
I20260809 00:37:55.732333 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.732242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58912)
0809 00:37:55.732277 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.732307 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 00:37:55.732426 15379 leader_election.cc:334] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213): Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
I20260809 00:37:55.732472 16005 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae79428707e4c7094364a1f6cefb8cf. Latest consensus state: current_term: 2 leader_uuid: "cae79428707e4c7094364a1f6cefb8cf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:55.732558 16005 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:55.732756 16007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:55.733253 16007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:55.733322 16007 catalog_manager.cc:1259] Loaded cluster ID: 0f35bb16ac2c4ea5b5cf1f077046c016
I20260809 00:37:55.733377 16007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:55.733534 16007 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:55.733628 16007 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Loaded TSK: 0
I20260809 00:37:55.733771 16007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:55.792165 15394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:55.794044 15394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:55.795145 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.795020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58930)
0809 00:37:55.795073 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.795110 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:37:55.795310 15193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [term 2 FOLLOWER]: Refusing update from remote peer cae79428707e4c7094364a1f6cefb8cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 00:37:55.795528 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.795431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.0.0.1:58936)
0809 00:37:55.795467 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.795498 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:37:55.795593 15993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 00:37:55.795702 15379 consensus_peers.cc:487] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf -> Peer 0a5aa01c47434b4586800d33a846ca77 (127.7.105.126:43213): Couldn't send request to peer 0a5aa01c47434b4586800d33a846ca77. Status: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 00:37:55.796285 15996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae79428707e4c7094364a1f6cefb8cf. Latest consensus state: current_term: 2 leader_uuid: "cae79428707e4c7094364a1f6cefb8cf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:55.796393 15996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:55.796867 15993 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cae79428707e4c7094364a1f6cefb8cf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:55.797025 15993 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:55.797202 15996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cae79428707e4c7094364a1f6cefb8cf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0a5aa01c47434b4586800d33a846ca77" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43213 } } peers { permanent_uuid: "ff0a616e88fa4dc69d8bce30e7d1f82c" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 38399 } attrs { promote: false } } peers { permanent_uuid: "cae79428707e4c7094364a1f6cefb8cf" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 35339 } attrs { promote: false } } }
I20260809 00:37:55.797300 15996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0a616e88fa4dc69d8bce30e7d1f82c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:55.801817 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38267 (local address 127.0.0.1:47832)
0809 00:37:55.798765 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.798776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.798825 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.799221 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.799222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.799224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.799276 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.799278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800254 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.800255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.800975 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.800977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.801622 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.801622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.801686 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.801690 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.801741 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.801771 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:37:55.801877 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.7.105.67:38267)
0809 00:37:55.799078 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.799080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.799092 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:37:55.799134 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.799135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.799145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.799216 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.799341 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800148 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.801075 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.801517 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.801589 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.801603 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.801766 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.801806 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260809 00:37:55.802079 15842 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8f8b672a374f538071d0512924d44c (DEFAULT_TABLE table=test-table [id=f05c50e866d84f288aecea996a62393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:55.802228 15842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8f8b672a374f538071d0512924d44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:55.802561 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.7.105.65:35213)
0809 00:37:55.798980 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.798982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.799218 (+ 236us) server_negotiation.cc:407] Connection header received
0809 00:37:55.799304 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.799305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.799315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.799375 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.799522 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800003 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.801407 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.801992 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.802062 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.802077 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.802448 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.802496 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260809 00:37:55.802829 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35505 (local address 127.0.0.1:56656)
0809 00:37:55.798736 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.798747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.798788 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.799139 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.799140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.799142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.799199 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.799201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800220 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.800221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.801316 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.801319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.802589 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.802590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.802678 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.802683 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.802723 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.802769 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":32}
I20260809 00:37:55.802884 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.7.105.66:35505)
0809 00:37:55.799003 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.799005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.799013 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:55.799042 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.799043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.799055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.799125 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.799363 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800144 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.802119 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.802484 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.802688 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.802704 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.802764 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.802806 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260809 00:37:55.803117 15693 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8f8b672a374f538071d0512924d44c (DEFAULT_TABLE table=test-table [id=f05c50e866d84f288aecea996a62393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:55.803162 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.798713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35213 (local address 127.0.0.1:45934)
0809 00:37:55.799161 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.799172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.799243 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.799426 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.799427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.799429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.799465 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.799466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.800110 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.800111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.801316 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.801318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.802214 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.802215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.802345 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.802352 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.802777 (+ 425us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.802829 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":45,"thread_start_us":72,"threads_started":1}
I20260809 00:37:55.803277 15693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8f8b672a374f538071d0512924d44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:55.804618 16024 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Bootstrap starting.
I20260809 00:37:55.804733 16023 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Bootstrap starting.
I20260809 00:37:55.804744 15527 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8f8b672a374f538071d0512924d44c (DEFAULT_TABLE table=test-table [id=f05c50e866d84f288aecea996a62393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:55.804870 15527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8f8b672a374f538071d0512924d44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:55.805004 16024 tablet_bootstrap.cc:654] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:55.805224 16023 tablet_bootstrap.cc:654] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:55.805655 16024 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: No bootstrap required, opened a new log
I20260809 00:37:55.805725 16024 ts_tablet_manager.cc:1397] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:55.805860 16024 raft_consensus.cc:348] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.805938 16024 raft_consensus.cc:374] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:55.805970 16024 raft_consensus.cc:729] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942f1a77b5f748eb997b6acd4ed2308b, State: Initialized, Role: FOLLOWER
I20260809 00:37:55.806025 16024 consensus_queue.cc:260] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.806334 16023 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: No bootstrap required, opened a new log
I20260809 00:37:55.806391 16023 ts_tablet_manager.cc:1397] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:55.806571 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.806373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:57649)
0809 00:37:55.806494 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.806532 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":81,"threads_started":1}
I20260809 00:37:55.806624 16023 raft_consensus.cc:348] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.806710 16023 raft_consensus.cc:374] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:55.806735 16023 raft_consensus.cc:729] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4622e06bd874a13bcfe35bc50ee7087, State: Initialized, Role: FOLLOWER
I20260809 00:37:55.806793 16023 consensus_queue.cc:260] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.806964 16023 ts_tablet_manager.cc:1428] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:55.807114 16024 ts_tablet_manager.cc:1428] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:55.807217 15765 heartbeater.cc:503] Master 127.7.105.124:35339 was elected leader, sending a full tablet report...
I20260809 00:37:55.807346 16030 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: Bootstrap starting.
I20260809 00:37:55.807638 15920 heartbeater.cc:503] Master 127.7.105.124:35339 was elected leader, sending a full tablet report...
I20260809 00:37:55.807780 16030 tablet_bootstrap.cc:654] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:55.808125 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.807986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:36629)
0809 00:37:55.808022 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.808097 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 00:37:55.808403 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.807157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:54309)
0809 00:37:55.808308 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.808366 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":87,"threads_started":1}
I20260809 00:37:55.808626 16030 tablet_bootstrap.cc:492] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: No bootstrap required, opened a new log
I20260809 00:37:55.808699 16030 ts_tablet_manager.cc:1397] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:55.808723 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.808619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:50943)
0809 00:37:55.808669 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.808699 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:37:55.808848 16030 raft_consensus.cc:348] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.808920 16030 raft_consensus.cc:374] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:55.808934 16030 raft_consensus.cc:729] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217efa1ae2894d0592fa498828bbd9d6, State: Initialized, Role: FOLLOWER
I20260809 00:37:55.808988 16030 consensus_queue.cc:260] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.809162 16030 ts_tablet_manager.cc:1428] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:55.809478 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.809300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:37275)
0809 00:37:55.809426 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.809455 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260809 00:37:55.809649 15609 heartbeater.cc:503] Master 127.7.105.124:35339 was elected leader, sending a full tablet report...
I20260809 00:37:55.809847 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.809764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:43123)
0809 00:37:55.809791 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.809817 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 00:37:55.840512 16037 raft_consensus.cc:482] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:55.840607 16037 raft_consensus.cc:504] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.840988 16037 leader_election.cc:288] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505), b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
W20260809 00:37:55.841334 15613 tablet.cc:2369] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:55.844558 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.841294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:52821 (local address 127.7.105.66:35505)
0809 00:37:55.841345 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.841348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.841473 (+ 125us) server_negotiation.cc:407] Connection header received
0809 00:37:55.841598 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.841598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.841610 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.841676 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.841848 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.842347 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.843417 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.844020 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.844158 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.844171 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.844431 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.844484 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:55.844566 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.841210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35505 (local address 127.7.105.65:52821)
0809 00:37:55.841394 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.841409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.841502 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.841679 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.841680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.841683 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.841774 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.841776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.842476 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.842477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.843335 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.843337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.844249 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.844250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.844334 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.844338 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.844406 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.844489 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260809 00:37:55.844621 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.841294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:45769 (local address 127.7.105.67:38267)
0809 00:37:55.841345 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.841348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.841576 (+ 228us) server_negotiation.cc:407] Connection header received
0809 00:37:55.841662 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.841665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.841679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.841752 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:55.842007 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.842511 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.843606 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.844200 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.844265 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.844276 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.844497 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.844546 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:37:55.844949 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.841210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38267 (local address 127.7.105.65:45769)
0809 00:37:55.841481 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.841493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:55.841589 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:55.841755 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:55.841756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:55.841758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:55.841854 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.841856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.842625 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.842626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.843516 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:55.843518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.844318 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.844320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:55.844427 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.844432 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:55.844488 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:55.844527 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":55,"thread_start_us":126,"threads_started":1}
I20260809 00:37:55.844961 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8f8b672a374f538071d0512924d44c" candidate_uuid: "217efa1ae2894d0592fa498828bbd9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942f1a77b5f748eb997b6acd4ed2308b" is_pre_election: true
I20260809 00:37:55.845054 15716 raft_consensus.cc:2454] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 217efa1ae2894d0592fa498828bbd9d6 in term 0.
I20260809 00:37:55.845255 15474 leader_election.cc:302] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 217efa1ae2894d0592fa498828bbd9d6, 942f1a77b5f748eb997b6acd4ed2308b; no voters:
I20260809 00:37:55.845276 15866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8f8b672a374f538071d0512924d44c" candidate_uuid: "217efa1ae2894d0592fa498828bbd9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4622e06bd874a13bcfe35bc50ee7087" is_pre_election: true
I20260809 00:37:55.845427 16037 raft_consensus.cc:2781] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:55.845440 15866 raft_consensus.cc:2454] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 217efa1ae2894d0592fa498828bbd9d6 in term 0.
I20260809 00:37:55.845475 16037 raft_consensus.cc:482] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:55.845512 16037 raft_consensus.cc:3037] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:55.846262 16037 raft_consensus.cc:504] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.846429 16037 leader_election.cc:288] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505), b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267)
I20260809 00:37:55.846586 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8f8b672a374f538071d0512924d44c" candidate_uuid: "217efa1ae2894d0592fa498828bbd9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942f1a77b5f748eb997b6acd4ed2308b"
I20260809 00:37:55.846627 15866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8f8b672a374f538071d0512924d44c" candidate_uuid: "217efa1ae2894d0592fa498828bbd9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4622e06bd874a13bcfe35bc50ee7087"
I20260809 00:37:55.846689 15716 raft_consensus.cc:3037] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:55.846697 15866 raft_consensus.cc:3037] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:55.847278 15716 raft_consensus.cc:2454] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 217efa1ae2894d0592fa498828bbd9d6 in term 1.
I20260809 00:37:55.847477 15866 raft_consensus.cc:2454] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 217efa1ae2894d0592fa498828bbd9d6 in term 1.
I20260809 00:37:55.847488 15474 leader_election.cc:302] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 217efa1ae2894d0592fa498828bbd9d6, 942f1a77b5f748eb997b6acd4ed2308b; no voters:
I20260809 00:37:55.847579 16037 raft_consensus.cc:2781] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:55.847754 16037 raft_consensus.cc:686] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [term 1 LEADER]: Becoming Leader. State: Replica: 217efa1ae2894d0592fa498828bbd9d6, State: Running, Role: LEADER
I20260809 00:37:55.847886 16037 consensus_queue.cc:237] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } }
I20260809 00:37:55.848155 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.848011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:58347)
0809 00:37:55.848092 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.848127 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 00:37:55.848696 15393 catalog_manager.cc:5495] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "217efa1ae2894d0592fa498828bbd9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "217efa1ae2894d0592fa498828bbd9d6" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:55.861763 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.858826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.7.105.65:35213)
0809 00:37:55.858888 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:55.858890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:55.858900 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:55.858973 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:55.858974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:55.858997 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:55.859084 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:55.859212 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.859810 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.860688 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:55.861070 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:55.861154 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:55.861338 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:55.861485 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:55.861553 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:55.861633 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:37:55.861654 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:37:55.863049 15866 raft_consensus.cc:1261] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Refusing update from remote peer 217efa1ae2894d0592fa498828bbd9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:55.863441 16040 consensus_queue.cc:1035] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4622e06bd874a13bcfe35bc50ee7087" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:55.864409 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.864259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:47383)
0809 00:37:55.864325 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.864371 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:37:55.865490 15716 raft_consensus.cc:1261] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Refusing update from remote peer 217efa1ae2894d0592fa498828bbd9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:55.865732 16040 consensus_queue.cc:1035] T 0c8f8b672a374f538071d0512924d44c P 217efa1ae2894d0592fa498828bbd9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "942f1a77b5f748eb997b6acd4ed2308b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:55.865978 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.865720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.65:59383)
0809 00:37:55.865916 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.865949 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260809 00:37:55.866528 16048 mvcc.cc:205] Tried to move back new op lower bound from 7316422147532402688 to 7316422147473543168. Current Snapshot: MvccSnapshot[applied={T|T < 7316422147532402688}]
I20260809 00:37:55.866566 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.866464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.67:49587)
0809 00:37:55.866504 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.866535 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 00:37:55.866832 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.866735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:39001)
0809 00:37:55.866769 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.866801 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 00:37:55.867329 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.867100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43213 (local address 127.7.105.66:41095)
0809 00:37:55.867273 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.867304 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:43213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 00:37:55.868618 15393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:55.868734 15393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:55.870210 15393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cae79428707e4c7094364a1f6cefb8cf: Sending DeleteTablet for 3 replicas of tablet 0c8f8b672a374f538071d0512924d44c
I20260809 00:37:55.870558 15693 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8f8b672a374f538071d0512924d44c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:55 UTC) from {username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:56656
I20260809 00:37:55.870777 15842 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8f8b672a374f538071d0512924d44c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:55 UTC) from {username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:47832
I20260809 00:37:55.870780 15527 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8f8b672a374f538071d0512924d44c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:55 UTC) from {username='oryx', principal='oryx/127.7.105.124@KRBTEST.COM'} at 127.0.0.1:45934
I20260809 00:37:55.871357 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 15349
I20260809 00:37:55.873240 16054 tablet_replica.cc:330] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: stopping tablet replica
I20260809 00:37:55.873298 16054 raft_consensus.cc:2229] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:55.873343 16054 raft_consensus.cc:2258] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:55.874580 16054 ts_tablet_manager.cc:1905] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:55.876473 16052 tablet_replica.cc:330] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: stopping tablet replica
I20260809 00:37:55.876536 16052 raft_consensus.cc:2229] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:55.876587 16052 raft_consensus.cc:2258] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:55.876897 16054 ts_tablet_manager.cc:1918] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:55.876952 16054 log.cc:1196] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/0c8f8b672a374f538071d0512924d44c
I20260809 00:37:55.877233 16054 ts_tablet_manager.cc:1939] T 0c8f8b672a374f538071d0512924d44c P b4622e06bd874a13bcfe35bc50ee7087: Deleting consensus metadata
I20260809 00:37:55.877563 15379 catalog_manager.cc:4841] TS b4622e06bd874a13bcfe35bc50ee7087 (127.7.105.67:38267): tablet 0c8f8b672a374f538071d0512924d44c (table test-table [id=f05c50e866d84f288aecea996a62393b]) successfully deleted
I20260809 00:37:55.878839 16052 ts_tablet_manager.cc:1905] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:55.880734 16052 ts_tablet_manager.cc:1918] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:55.880790 16052 log.cc:1196] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/0c8f8b672a374f538071d0512924d44c
I20260809 00:37:55.881021 16052 ts_tablet_manager.cc:1939] T 0c8f8b672a374f538071d0512924d44c P 942f1a77b5f748eb997b6acd4ed2308b: Deleting consensus metadata
I20260809 00:37:55.881636 15378 catalog_manager.cc:4841] TS 942f1a77b5f748eb997b6acd4ed2308b (127.7.105.66:35505): tablet 0c8f8b672a374f538071d0512924d44c (table test-table [id=f05c50e866d84f288aecea996a62393b]) successfully deleted
I20260809 00:37:55.884877 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 15620
I20260809 00:37:55.886732 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:55.886600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35213 (local address 127.0.0.1:45964)
0809 00:37:55.886658 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:55.886686 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:37:55.887082 15377 catalog_manager.cc:4584] TS 217efa1ae2894d0592fa498828bbd9d6 (127.7.105.65:35213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c8f8b672a374f538071d0512924d44c: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35213: connect: Connection refused (error 111)
I20260809 00:37:55.894531 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 15773
I20260809 00:37:55.904671 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 15025
I20260809 00:37:55.914855 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 15245
2026-08-09T00:37:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4764 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:55 dist-test-slave-1zqn krb5kdc[16058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:55 dist-test-slave-1zqn krb5kdc[16058](info): set up 2 sockets
Aug 09 00:37:55 dist-test-slave-1zqn krb5kdc[16058](info): commencing operation
krb5kdc: starting...
W20260809 00:37:57.020783 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab.
Entry for principal oryx/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab.
I20260809 00:37:57.152549 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44349
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab
--principal=oryx/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:57.235805 16073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:57.236033 16073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:57.236074 16073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:57.237314 16073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:57.237360 16073 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:37:57.237373 16073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:57.237386 16073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:57.237399 16073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:57.237411 16073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:57.238972 16073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44349
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.126.keytab
--principal=oryx/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:57.239172 16073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:57.239343 16073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:57.244524 16078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.244549 16081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.244573 16079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:57.244854 16073 server_base.cc:1034] running on GCE node
I20260809 00:37:57.245213 16073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:57.245509 16073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:57.246673 16073 hybrid_clock.cc:648] HybridClock initialized: now 1786235877246672 us; error 41 us; skew 500 ppm
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:57.249374 16073 init.cc:377] Logged in from keytab as oryx/127.7.105.126@KRBTEST.COM (short username oryx)
I20260809 00:37:57.249856 16073 webserver.cc:460] Webserver started at http://127.7.105.126:36277/ using document root <none> and password file <none>
I20260809 00:37:57.250084 16073 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:57.250154 16073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:57.250305 16073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:57.251721 16073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "7b5773ead6ae42548a5ffc121e32ce7b"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "c9188bc2a1f36604c30e8fd622ee591c"
server_key_iv: "da07a101bba22a194ffe329782586692"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.252167 16073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "7b5773ead6ae42548a5ffc121e32ce7b"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "c9188bc2a1f36604c30e8fd622ee591c"
server_key_iv: "da07a101bba22a194ffe329782586692"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.253649 16073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:57.254416 16090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.254585 16073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:57.254684 16073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "7b5773ead6ae42548a5ffc121e32ce7b"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "c9188bc2a1f36604c30e8fd622ee591c"
server_key_iv: "da07a101bba22a194ffe329782586692"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.254768 16073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:57.262355 16073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:57.264650 16073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:57.264809 16073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:57.270053 16073 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44349
I20260809 00:37:57.270140 16161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44349 every 8 connection(s)
I20260809 00:37:57.270499 16073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:57.271229 16162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.273545 16162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Bootstrap starting.
I20260809 00:37:57.274107 16162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.274358 16162 log.cc:824] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:57.275039 16162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: No bootstrap required, opened a new log
I20260809 00:37:57.277160 16162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } }
I20260809 00:37:57.277297 16162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.277321 16162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b5773ead6ae42548a5ffc121e32ce7b, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.277455 16162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } }
I20260809 00:37:57.277539 16162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:57.277561 16162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:57.277590 16162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:57.278080 16162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } }
I20260809 00:37:57.278180 16162 leader_election.cc:302] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b5773ead6ae42548a5ffc121e32ce7b; no voters:
I20260809 00:37:57.278388 16162 leader_election.cc:288] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:57.278364 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 16073
I20260809 00:37:57.278462 16165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:57.278491 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:37:57.278618 16165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [term 1 LEADER]: Becoming Leader. State: Replica: 7b5773ead6ae42548a5ffc121e32ce7b, State: Running, Role: LEADER
I20260809 00:37:57.278666 16162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:57.278733 7589 external_mini_cluster.cc:1428] Setting key e332a1e88bd94c2ee924a5fc08c47336
I20260809 00:37:57.278748 16165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } }
I20260809 00:37:57.279749 16166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } } }
I20260809 00:37:57.279876 16166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:57.279739 16167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b5773ead6ae42548a5ffc121e32ce7b. Latest consensus state: current_term: 1 leader_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5773ead6ae42548a5ffc121e32ce7b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44349 } } }
I20260809 00:37:57.280097 16167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:57.280113 16185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:57.280965 16185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:57.282639 16185 catalog_manager.cc:1347] Generated new cluster ID: 716eae12744141998692d9d5ce3409ae
I20260809 00:37:57.282693 16185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:57.289992 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.279304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.7.105.126:44349)
0809 00:37:57.279484 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.279494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.279627 (+ 133us) server_negotiation.cc:407] Connection header received
0809 00:37:57.279771 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.279777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.279834 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.280999 (+ 1165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.281094 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.282290 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.283231 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.283682 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.283804 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.286255 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.286259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.286259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.286272 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.288398 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.288797 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.288800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.288801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.288896 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.289120 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.289121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.289121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.289372 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.289567 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.289634 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.289775 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260809 00:37:57.296084 16185 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:57.296638 16185 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:57.308209 16185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Generated new TSK 0
I20260809 00:37:57.308408 16185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
I20260809 00:37:57.339931 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44349
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:57.426290 16192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:57.426499 16192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:57.426525 16192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:57.427623 16192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:57.427661 16192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:57.427682 16192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:57.427700 16192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:37:57.429281 16192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:57.429498 16192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:57.429692 16192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:57.435279 16199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.435220 16202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.435218 16197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:57.435252 16192 server_base.cc:1034] running on GCE node
I20260809 00:37:57.435756 16192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:57.436028 16192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:57.437237 16192 hybrid_clock.cc:648] HybridClock initialized: now 1786235877437227 us; error 38 us; skew 500 ppm
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:57.440022 16192 init.cc:377] Logged in from keytab as oryx/127.7.105.65@KRBTEST.COM (short username oryx)
I20260809 00:37:57.440490 16192 webserver.cc:460] Webserver started at http://127.7.105.65:42905/ using document root <none> and password file <none>
I20260809 00:37:57.440713 16192 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:57.440757 16192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:57.440897 16192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:57.442041 16192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "ea6771fa1462499abda52ce0f673a715"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "fb59afda53c809625539f418e8d75fc8"
server_key_iv: "e2788980886365218672105f5b26a5ce"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.442400 16192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "ea6771fa1462499abda52ce0f673a715"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "fb59afda53c809625539f418e8d75fc8"
server_key_iv: "e2788980886365218672105f5b26a5ce"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.443801 16192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:57.444715 16211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.444898 16192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:57.445011 16192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "ea6771fa1462499abda52ce0f673a715"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "fb59afda53c809625539f418e8d75fc8"
server_key_iv: "e2788980886365218672105f5b26a5ce"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.445091 16192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:57.458508 16192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:57.461081 16192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:57.461257 16192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:57.461809 16192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:57.461869 16192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.461905 16192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:57.461961 16192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:57.469807 16192 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35017
I20260809 00:37:57.469898 16354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35017 every 8 connection(s)
I20260809 00:37:57.470221 16192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:37:57.473246 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.462486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:40559 (local address 127.7.105.126:44349)
0809 00:37:57.462561 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.462564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.462835 (+ 271us) server_negotiation.cc:407] Connection header received
0809 00:37:57.463585 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.463587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.463603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.463669 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.464689 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.465288 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.466586 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.466840 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.466953 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.469729 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.469731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.469732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.469741 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.471581 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.472083 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.472085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.472085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.472153 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.472450 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.472451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.472452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.472589 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.472672 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.473052 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.473145 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:37:57.473460 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.462469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.65:40559)
0809 00:37:57.462669 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.462711 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.463430 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.463795 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.463801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.464252 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.464567 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.464576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.465402 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.465403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.466475 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.466478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.466973 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.466975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.467178 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.467753 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.467762 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.469613 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.471698 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.471706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.471709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.471969 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.472237 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.472238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.472238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.472365 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.472662 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.472670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.472859 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.473012 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.473193 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260809 00:37:57.473992 16355 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44349
I20260809 00:37:57.474087 16355 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:57.474294 16355 heartbeater.cc:511] Master 127.7.105.126:44349 requested a full tablet report, sending...
I20260809 00:37:57.474859 16108 ts_manager.cc:194] Registered new tserver with Master: ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017)
I20260809 00:37:57.475548 16108 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:40559
I20260809 00:37:57.475888 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 16192
I20260809 00:37:57.476027 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:37:57.476294 7589 external_mini_cluster.cc:1428] Setting key d17385f079e223487f13de32c2fd75e2
I20260809 00:37:57.482043 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.475236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:45393 (local address 127.7.105.126:44349)
0809 00:37:57.475303 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.475305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.475361 (+ 56us) server_negotiation.cc:407] Connection header received
0809 00:37:57.475439 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.475440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.475452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.475500 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.475990 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.476719 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.477781 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.477960 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.478012 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.479187 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.479189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.479189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.479198 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.480863 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.481303 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.481305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.481305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.481355 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.481575 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.481575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.481576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.481672 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.481723 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.481912 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.481977 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:37:57.482216 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.475171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.65:45393)
0809 00:37:57.475304 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.475317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.475377 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.475515 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.475515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.475780 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.475872 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.475874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.476834 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.476835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.477683 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.477687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.478036 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.478038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.478127 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.478588 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.478594 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.479093 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.480953 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.480954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.480955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.481218 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.481418 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.481419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.481420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.481509 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.481736 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.481738 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.481797 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.481889 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.482036 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
I20260809 00:37:57.507383 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44349
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:57.591804 16367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:57.592023 16367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:57.592048 16367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:57.593286 16367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:57.593331 16367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:57.593353 16367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:57.593370 16367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:37:57.594934 16367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:57.595125 16367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:57.595381 16367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:57.600562 16373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.600570 16376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:57.600759 16367 server_base.cc:1034] running on GCE node
W20260809 00:37:57.600963 16374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:57.601235 16367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:57.601534 16367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:57.602712 16367 hybrid_clock.cc:648] HybridClock initialized: now 1786235877602695 us; error 31 us; skew 500 ppm
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:57.605459 16367 init.cc:377] Logged in from keytab as oryx/127.7.105.66@KRBTEST.COM (short username oryx)
I20260809 00:37:57.606035 16367 webserver.cc:460] Webserver started at http://127.7.105.66:38119/ using document root <none> and password file <none>
I20260809 00:37:57.606299 16367 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:57.606380 16367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:57.606563 16367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:57.607837 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "af0bf2be30a0411686b22a864c772cb8"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "38b39fa3683abda14c23c1eab85f5f56"
server_key_iv: "0f9719f57fee7be0d39a2bd4df02591d"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.608290 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "af0bf2be30a0411686b22a864c772cb8"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "38b39fa3683abda14c23c1eab85f5f56"
server_key_iv: "0f9719f57fee7be0d39a2bd4df02591d"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.609772 16367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:57.610656 16386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.610836 16367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:57.611027 16367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "af0bf2be30a0411686b22a864c772cb8"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "38b39fa3683abda14c23c1eab85f5f56"
server_key_iv: "0f9719f57fee7be0d39a2bd4df02591d"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.611123 16367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:57.621687 16367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:57.623848 16367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:57.624037 16367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:57.624768 16367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:57.624830 16367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.624886 16367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:57.624914 16367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:57.632817 16367 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35861
I20260809 00:37:57.632907 16530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35861 every 8 connection(s)
I20260809 00:37:57.633291 16367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:37:57.635596 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.625269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41317 (local address 127.7.105.126:44349)
0809 00:37:57.625343 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.625346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.625609 (+ 263us) server_negotiation.cc:407] Connection header received
0809 00:37:57.626375 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.626376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.626387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.626426 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.627377 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.627992 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.629734 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.629924 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.630014 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.632751 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.632757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.632757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.632765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.634299 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.634748 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.634749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.634750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.634808 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.635063 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.635064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.635064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.635174 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.635228 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.635466 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.635530 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:37:57.635809 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.625249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.66:41317)
0809 00:37:57.625453 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.625510 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.626252 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.626531 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.626535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.626961 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.627283 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.627292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.628139 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.628141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.629636 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.629640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.630067 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.630068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.630222 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.630754 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.630763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.632616 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.634397 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.634404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.634406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.634666 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.634871 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.634872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.634872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.634983 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.635220 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.635222 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.635349 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.635431 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.635574 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260809 00:37:57.636379 16531 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44349
I20260809 00:37:57.636468 16531 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:57.636648 16531 heartbeater.cc:511] Master 127.7.105.126:44349 requested a full tablet report, sending...
I20260809 00:37:57.637081 16109 ts_manager.cc:194] Registered new tserver with Master: af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861)
I20260809 00:37:57.637710 16109 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:41317
I20260809 00:37:57.643440 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.637621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:33449 (local address 127.7.105.126:44349)
0809 00:37:57.637665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.637667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.637746 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:37:57.637814 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.637815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.637825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.637885 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.638253 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.638804 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.639625 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.639858 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.639943 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.640888 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.640889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.640890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.640897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.642168 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.642569 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.642571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.642572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.642635 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.642844 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.642845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.642845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.643005 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.643100 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.643278 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.643348 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:57.643447 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.637524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.66:33449)
0809 00:37:57.637674 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.637687 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.637758 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.637891 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.637892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.638085 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.638153 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.638155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.638921 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.638922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.639508 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.639510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.639951 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.639953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.640030 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.640488 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.640493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.640780 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.642283 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.642285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.642285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.642463 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.642713 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.642713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.642714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.642779 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.643092 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.643093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.643142 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.643231 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.643348 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260809 00:37:57.643592 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 16367
I20260809 00:37:57.643726 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:37:57.643935 7589 external_mini_cluster.cc:1428] Setting key 1299b5894210978b6609ebc09275757c
WARNING: no policy specified for oryx/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
I20260809 00:37:57.672401 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44349
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:57.756042 16540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:57.756315 16540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:57.756340 16540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:57.757572 16540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:57.757613 16540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:57.757634 16540 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:37:57.757651 16540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:37:57.759220 16540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.105.126:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:57.759413 16540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:57.759589 16540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:57.765031 16549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.765141 16547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:57.765034 16546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:57.765260 16540 server_base.cc:1034] running on GCE node
I20260809 00:37:57.765539 16540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:57.765880 16540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:57.767058 16540 hybrid_clock.cc:648] HybridClock initialized: now 1786235877767050 us; error 53 us; skew 500 ppm
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:57.769923 16540 init.cc:377] Logged in from keytab as oryx/127.7.105.67@KRBTEST.COM (short username oryx)
I20260809 00:37:57.770457 16540 webserver.cc:460] Webserver started at http://127.7.105.67:37815/ using document root <none> and password file <none>
I20260809 00:37:57.770704 16540 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:57.770776 16540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:57.770927 16540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:57.772010 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "89627bd75a53413995bbd8036bcf3cf5"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "cc2bff805c14d5bcbeab5972dbfb510d"
server_key_iv: "5b828d4f62524defe8022265d191d82f"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.772567 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "89627bd75a53413995bbd8036bcf3cf5"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "cc2bff805c14d5bcbeab5972dbfb510d"
server_key_iv: "5b828d4f62524defe8022265d191d82f"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.773903 16540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:57.774724 16557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.774933 16540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:57.775033 16540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "89627bd75a53413995bbd8036bcf3cf5"
format_stamp: "Formatted at 2026-08-09 00:37:57 on dist-test-slave-1zqn"
server_key: "cc2bff805c14d5bcbeab5972dbfb510d"
server_key_iv: "5b828d4f62524defe8022265d191d82f"
server_key_version: "encryptionkey@0"
I20260809 00:37:57.775111 16540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:57.796793 16540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:57.799304 16540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:57.799484 16540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:57.800107 16540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:37:57.800176 16540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.800212 16540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:37:57.800241 16540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:57.807749 16540 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:42359
I20260809 00:37:57.807909 16692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:42359 every 8 connection(s)
I20260809 00:37:57.808357 16540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:37:57.810871 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.800619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:39367 (local address 127.7.105.126:44349)
0809 00:37:57.800696 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.800699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.800979 (+ 280us) server_negotiation.cc:407] Connection header received
0809 00:37:57.801725 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.801727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.801737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.801784 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.802719 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.803547 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.805309 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.805513 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.805601 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.808241 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.808243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.808243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.808250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.809619 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.810107 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.810108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.810108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.810161 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.810377 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.810377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.810377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.810477 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.810527 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.810744 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.810807 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:37:57.811141 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.800599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.67:39367)
0809 00:37:57.800810 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.800863 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.801612 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.801869 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.801874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.802300 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.802613 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.802620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.803659 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.803660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.805188 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.805192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.805610 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.805611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.805743 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.806342 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.806351 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.808155 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.809697 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.809703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.809705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.810025 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.810220 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.810221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.810221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.810322 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.810532 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.810537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.810651 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.810732 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.810856 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260809 00:37:57.811672 16693 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44349
I20260809 00:37:57.811760 16693 heartbeater.cc:464] Registering TS with master...
I20260809 00:37:57.811959 16693 heartbeater.cc:511] Master 127.7.105.126:44349 requested a full tablet report, sending...
I20260809 00:37:57.812399 16108 ts_manager.cc:194] Registered new tserver with Master: 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:57.812805 16108 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:39367
I20260809 00:37:57.818239 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 16540
I20260809 00:37:57.818359 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:37:57.818617 7589 external_mini_cluster.cc:1428] Setting key e601d5aa763eff9694817358f1d17b27
I20260809 00:37:57.819209 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.812814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.7.105.67:53155)
0809 00:37:57.812972 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.812982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.813075 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.813187 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.813188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.813404 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:57.813467 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.813469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.814141 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.814141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.814744 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.814746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.815399 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.815400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.815481 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.815890 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:57.815894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:57.816246 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:57.817983 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.817984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.817984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.818162 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.818446 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:57.818447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:57.818448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:57.818520 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:57.818844 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:57.818845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:57.818898 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.818977 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.819065 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260809 00:37:57.819279 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.812868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:53155 (local address 127.7.105.126:44349)
0809 00:37:57.812914 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.812916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.813051 (+ 135us) server_negotiation.cc:407] Connection header received
0809 00:37:57.813123 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.813124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.813135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.813185 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.813554 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.814050 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.815053 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.815256 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.815376 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.816331 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.816333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.816333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.816340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.817899 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.818245 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.818246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.818246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.818320 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.818606 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.818607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.818608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.818764 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.818847 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.819066 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.819177 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:57.819737 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:37:57.845181 16174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:57.838885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.7.105.126:44349)
0809 00:37:57.838959 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.838967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.838978 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:57.839004 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.839005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.839016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.839067 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.839666 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.840337 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.841201 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.841388 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.841447 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.845069 (+ 3622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57882: BlockingRecv error: recv got EOF from 127.0.0.1:57882 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 00:37:57 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:57.853343 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.846182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.7.105.126:44349)
0809 00:37:57.846250 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.846252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.846263 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:57.846304 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.846305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.846314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.846361 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:57.846813 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.847446 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.848312 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.848552 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.848616 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.850822 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:57.850824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:57.850824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:57.850831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:57.852382 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.852690 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.852692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.852692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.852745 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:57.852911 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:57.852911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:57.852912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:57.853048 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:57.853129 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.853215 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.853271 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:37:57.855170 16109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:57.856963 16109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:57.863891 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.7.105.67:42359)
0809 00:37:57.860209 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.860213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.860369 (+ 156us) server_negotiation.cc:407] Connection header received
0809 00:37:57.860419 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.860421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.860436 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.860517 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:57.860601 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861656 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.862941 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863446 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863537 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863622 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.863751 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.863803 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":85,"threads_started":1}
I20260809 00:37:57.863885 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42359 (local address 127.0.0.1:45828)
0809 00:37:57.860261 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.860270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.860309 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.860514 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.860515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.860517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:57.860546 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.860547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861736 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.861737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.862844 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.862846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863538 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.863623 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863638 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.863734 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.863794 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260809 00:37:57.863983 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35861 (local address 127.0.0.1:53600)
0809 00:37:57.860128 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.860139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.860200 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.860472 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.860473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.860476 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:57.860533 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.860535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861458 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.861459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.862744 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.862747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863722 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.863809 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863814 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.863866 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.863912 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260809 00:37:57.864118 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.7.105.65:35017)
0809 00:37:57.859872 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.859877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.859888 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:57.859933 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.859935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.859953 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.860037 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:57.860436 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861599 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.862730 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863476 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863591 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863679 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.863912 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.863956 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260809 00:37:57.864319 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.0.0.1:45622)
0809 00:37:57.859713 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.859755 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.859824 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.860117 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.860120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.860123 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:57.860339 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.860341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861708 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.861709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.862646 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.862648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863721 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.863816 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863819 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.864207 (+ 388us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.864247 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":23,"thread_start_us":61,"threads_started":1}
I20260809 00:37:57.864336 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.859819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.7.105.66:35861)
0809 00:37:57.859997 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.860000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.860201 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:37:57.860300 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.860303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.860319 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.860401 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:57.860629 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.861363 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.862849 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.863569 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.863628 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.863694 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.864187 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.864233 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260809 00:37:57.864996 16452 tablet_service.cc:1467] Processing CreateTablet for tablet 4341ce1236c94e13a1eee9deba24d64b (DEFAULT_TABLE table=test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.865034 16618 tablet_service.cc:1467] Processing CreateTablet for tablet 4341ce1236c94e13a1eee9deba24d64b (DEFAULT_TABLE table=test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.865034 16276 tablet_service.cc:1467] Processing CreateTablet for tablet 4341ce1236c94e13a1eee9deba24d64b (DEFAULT_TABLE table=test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.865311 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4341ce1236c94e13a1eee9deba24d64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.865377 16618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4341ce1236c94e13a1eee9deba24d64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.865377 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4341ce1236c94e13a1eee9deba24d64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.868453 16726 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Bootstrap starting.
I20260809 00:37:57.869033 16726 tablet_bootstrap.cc:654] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.869234 16727 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Bootstrap starting.
I20260809 00:37:57.869380 16726 log.cc:824] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:57.869642 16728 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Bootstrap starting.
I20260809 00:37:57.869935 16727 tablet_bootstrap.cc:654] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.870263 16728 tablet_bootstrap.cc:654] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.870280 16727 log.cc:824] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:57.870362 16726 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: No bootstrap required, opened a new log
I20260809 00:37:57.870436 16726 ts_tablet_manager.cc:1397] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:57.870627 16728 log.cc:824] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:57.871405 16727 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: No bootstrap required, opened a new log
I20260809 00:37:57.871487 16727 ts_tablet_manager.cc:1397] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:57.871743 16728 tablet_bootstrap.cc:492] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: No bootstrap required, opened a new log
I20260809 00:37:57.871816 16728 ts_tablet_manager.cc:1397] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:57.872432 16726 raft_consensus.cc:348] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.872596 16726 raft_consensus.cc:374] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.872640 16726 raft_consensus.cc:729] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.872756 16726 consensus_queue.cc:260] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.873046 16726 ts_tablet_manager.cc:1428] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:57.873070 16531 heartbeater.cc:503] Master 127.7.105.126:44349 was elected leader, sending a full tablet report...
I20260809 00:37:57.873576 16727 raft_consensus.cc:348] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.873787 16727 raft_consensus.cc:374] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.873842 16727 raft_consensus.cc:729] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.873840 16728 raft_consensus.cc:348] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.873991 16728 raft_consensus.cc:374] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.873975 16727 consensus_queue.cc:260] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.874035 16728 raft_consensus.cc:729] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.874145 16728 consensus_queue.cc:260] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.874266 16727 ts_tablet_manager.cc:1428] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:37:57.874303 16693 heartbeater.cc:503] Master 127.7.105.126:44349 was elected leader, sending a full tablet report...
I20260809 00:37:57.874415 16728 ts_tablet_manager.cc:1428] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:37:57.874624 16355 heartbeater.cc:503] Master 127.7.105.126:44349 was elected leader, sending a full tablet report...
W20260809 00:37:57.883870 16532 tablet.cc:2369] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:57.885433 16735 raft_consensus.cc:482] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:57.885519 16735 raft_consensus.cc:504] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.885898 16735 leader_election.cc:288] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861)
I20260809 00:37:57.890038 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.886248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:39789 (local address 127.7.105.66:35861)
0809 00:37:57.886292 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.886294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.886304 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:57.886389 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.886390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.886400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.886450 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:57.886659 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.887526 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.888992 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.889623 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.889711 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.889720 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.889917 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.889971 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:37:57.890014 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.886174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35861 (local address 127.7.105.67:39789)
0809 00:37:57.886225 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.886238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.886309 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.886457 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.886458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.886460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:57.886528 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.886530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.887615 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.887616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.888886 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.888890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.889714 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.889715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.889801 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.889802 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.889863 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.889909 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":10}
I20260809 00:37:57.890066 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.886322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:55479 (local address 127.7.105.65:35017)
0809 00:37:57.886367 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.886369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.886641 (+ 272us) server_negotiation.cc:407] Connection header received
0809 00:37:57.886677 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.886678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.886690 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.886756 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:57.886880 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.887672 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.888992 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.889623 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.889711 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.889720 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.889954 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.890000 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:57.890015 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.886181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.7.105.67:55479)
0809 00:37:57.886560 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:57.886571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:57.886626 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:57.886764 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:57.886764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:57.886766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:57.886818 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.886820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.887742 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.887742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.888892 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:57.888894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.889714 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.889715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:57.889798 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.889802 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:57.889863 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:37:57.889909 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":213,"thread_start_us":136,"threads_started":1}
I20260809 00:37:57.890375 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4341ce1236c94e13a1eee9deba24d64b" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8" is_pre_election: true
I20260809 00:37:57.890538 16475 raft_consensus.cc:2454] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 0.
I20260809 00:37:57.890636 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4341ce1236c94e13a1eee9deba24d64b" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715" is_pre_election: true
I20260809 00:37:57.890787 16298 raft_consensus.cc:2454] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 0.
I20260809 00:37:57.890775 16561 leader_election.cc:302] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, af0bf2be30a0411686b22a864c772cb8; no voters:
I20260809 00:37:57.890897 16735 raft_consensus.cc:2781] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:57.891003 16735 raft_consensus.cc:482] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:57.891044 16735 raft_consensus.cc:3037] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:57.891963 16735 raft_consensus.cc:504] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.892144 16735 leader_election.cc:288] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 election: Requested vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861)
I20260809 00:37:57.892297 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4341ce1236c94e13a1eee9deba24d64b" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8"
I20260809 00:37:57.892307 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4341ce1236c94e13a1eee9deba24d64b" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715"
I20260809 00:37:57.892395 16475 raft_consensus.cc:3037] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:57.892393 16298 raft_consensus.cc:3037] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:57.893087 16475 raft_consensus.cc:2454] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 1.
I20260809 00:37:57.893151 16298 raft_consensus.cc:2454] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 1.
I20260809 00:37:57.893344 16558 leader_election.cc:302] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:57.893452 16735 raft_consensus.cc:2781] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:57.893631 16735 raft_consensus.cc:686] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 1 LEADER]: Becoming Leader. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Running, Role: LEADER
I20260809 00:37:57.893738 16735 consensus_queue.cc:237] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } }
I20260809 00:37:57.894601 16108 catalog_manager.cc:5495] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "89627bd75a53413995bbd8036bcf3cf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:57.903227 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:57.901006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.7.105.67:42359)
0809 00:37:57.901071 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:57.901073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:57.901081 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:57.901110 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:57.901110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:57.901127 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:57.901187 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:57.901320 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.901788 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.902512 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:57.902700 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:57.902773 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:57.902936 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:57.903037 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:57.903117 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:57.903138 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 00:37:57.903160 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:37:57.905336 16298 raft_consensus.cc:1261] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Refusing update from remote peer 89627bd75a53413995bbd8036bcf3cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:57.905433 16475 raft_consensus.cc:1261] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Refusing update from remote peer 89627bd75a53413995bbd8036bcf3cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:57.905725 16738 consensus_queue.cc:1035] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:57.905869 16735 consensus_queue.cc:1035] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:57.911231 16109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:57.911358 16109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:57.912256 16109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Sending DeleteTablet for 3 replicas of tablet 4341ce1236c94e13a1eee9deba24d64b
I20260809 00:37:57.912626 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet 4341ce1236c94e13a1eee9deba24d64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:57 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45622
I20260809 00:37:57.912956 16452 tablet_service.cc:1514] Processing DeleteTablet for tablet 4341ce1236c94e13a1eee9deba24d64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:57 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:53600
I20260809 00:37:57.913008 16109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:37:57.913079 16618 tablet_service.cc:1514] Processing DeleteTablet for tablet 4341ce1236c94e13a1eee9deba24d64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:57 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45828
W20260809 00:37:57.913164 16109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:57.913410 16755 tablet_replica.cc:330] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: stopping tablet replica
I20260809 00:37:57.913532 16755 raft_consensus.cc:2229] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:57.913667 16755 raft_consensus.cc:2258] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:57.914606 16618 tablet_service.cc:1467] Processing CreateTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 (DEFAULT_TABLE table=test-table [id=d76df6d9eb374017ada30168374c0199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.914741 16618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2797cd1eac417886d0fc6580e1e5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.914813 16755 ts_tablet_manager.cc:1905] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:57.915046 16276 tablet_service.cc:1467] Processing CreateTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 (DEFAULT_TABLE table=test-table [id=d76df6d9eb374017ada30168374c0199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.915158 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2797cd1eac417886d0fc6580e1e5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.916200 16727 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Bootstrap starting.
I20260809 00:37:57.916604 16728 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Bootstrap starting.
I20260809 00:37:57.916692 16727 tablet_bootstrap.cc:654] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.916743 16452 tablet_service.cc:1467] Processing CreateTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 (DEFAULT_TABLE table=test-table [id=d76df6d9eb374017ada30168374c0199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:57.916908 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2797cd1eac417886d0fc6580e1e5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:57.917052 16728 tablet_bootstrap.cc:654] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.917413 16756 tablet_replica.cc:330] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: stopping tablet replica
I20260809 00:37:57.917528 16756 raft_consensus.cc:2229] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:57.917658 16756 raft_consensus.cc:2258] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:57.918360 16755 ts_tablet_manager.cc:1918] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:57.918458 16755 log.cc:1196] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/4341ce1236c94e13a1eee9deba24d64b
I20260809 00:37:57.918452 16726 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Bootstrap starting.
I20260809 00:37:57.918784 16755 ts_tablet_manager.cc:1939] T 4341ce1236c94e13a1eee9deba24d64b P ea6771fa1462499abda52ce0f673a715: Deleting consensus metadata
I20260809 00:37:57.918900 16726 tablet_bootstrap.cc:654] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:57.919224 16091 catalog_manager.cc:4841] TS ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017): tablet 4341ce1236c94e13a1eee9deba24d64b (table test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]) successfully deleted
I20260809 00:37:57.919516 16727 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: No bootstrap required, opened a new log
I20260809 00:37:57.919561 16727 ts_tablet_manager.cc:1397] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:37:57.919602 16728 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: No bootstrap required, opened a new log
I20260809 00:37:57.919651 16728 ts_tablet_manager.cc:1397] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 00:37:57.919697 16727 raft_consensus.cc:348] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.919763 16727 raft_consensus.cc:374] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.919780 16727 raft_consensus.cc:729] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.919797 16728 raft_consensus.cc:348] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.919858 16728 raft_consensus.cc:374] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.919879 16728 raft_consensus.cc:729] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.919852 16727 consensus_queue.cc:260] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.919927 16728 consensus_queue.cc:260] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.919963 16727 ts_tablet_manager.cc:1428] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.920010 16728 ts_tablet_manager.cc:1428] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.921165 16756 ts_tablet_manager.cc:1905] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:57.922583 16756 ts_tablet_manager.cc:1918] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:57.922653 16756 log.cc:1196] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/4341ce1236c94e13a1eee9deba24d64b
I20260809 00:37:57.922851 16756 ts_tablet_manager.cc:1939] T 4341ce1236c94e13a1eee9deba24d64b P af0bf2be30a0411686b22a864c772cb8: Deleting consensus metadata
I20260809 00:37:57.923287 16094 catalog_manager.cc:4841] TS af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861): tablet 4341ce1236c94e13a1eee9deba24d64b (table test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]) successfully deleted
I20260809 00:37:57.924175 16757 tablet_replica.cc:330] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: stopping tablet replica
I20260809 00:37:57.924288 16757 raft_consensus.cc:2229] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:57.924391 16757 raft_consensus.cc:2258] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:57.924505 16726 tablet_bootstrap.cc:492] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: No bootstrap required, opened a new log
I20260809 00:37:57.924561 16726 ts_tablet_manager.cc:1397] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 00:37:57.924719 16726 raft_consensus.cc:348] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.924799 16726 raft_consensus.cc:374] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:57.924823 16726 raft_consensus.cc:729] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Initialized, Role: FOLLOWER
I20260809 00:37:57.924876 16726 consensus_queue.cc:260] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:57.925038 16726 ts_tablet_manager.cc:1428] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:57.925524 16757 ts_tablet_manager.cc:1905] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:57.927136 16757 ts_tablet_manager.cc:1918] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:57.927201 16757 log.cc:1196] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/4341ce1236c94e13a1eee9deba24d64b
I20260809 00:37:57.927451 16757 ts_tablet_manager.cc:1939] T 4341ce1236c94e13a1eee9deba24d64b P 89627bd75a53413995bbd8036bcf3cf5: Deleting consensus metadata
I20260809 00:37:57.928153 16092 catalog_manager.cc:4841] TS 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359): tablet 4341ce1236c94e13a1eee9deba24d64b (table test-table [id=8957c35ef69a44a7a70b7cd0dfba36a6]) successfully deleted
W20260809 00:37:57.971041 16356 tablet.cc:2369] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:37:58.032449 16734 raft_consensus.cc:482] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:58.032572 16734 raft_consensus.cc:504] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.032982 16734 leader_election.cc:288] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.036209 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.033248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:36231 (local address 127.7.105.65:35017)
0809 00:37:58.033295 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.033297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.033308 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:58.033417 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.033418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.033428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.033482 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:58.033693 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.034357 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035393 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.035771 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035862 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.035872 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.036043 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.036121 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:37:58.036221 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.033190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.7.105.66:36231)
0809 00:37:58.033255 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.033267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.033316 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.033535 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.033536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.033538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:58.033616 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.033618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.034453 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.034454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.035296 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.035298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.035853 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.035969 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.035973 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.036031 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.036119 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":69}
I20260809 00:37:58.036321 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.033482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35265 (local address 127.7.105.67:42359)
0809 00:37:58.033533 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.033536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.033546 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:58.033580 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.033581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.033592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.033673 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:58.033957 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.034477 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035491 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.035857 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035957 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.035967 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.036194 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.036247 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:58.036387 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.033277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42359 (local address 127.7.105.66:35265)
0809 00:37:58.033466 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.033477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.033524 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.033737 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.033738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.033740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:58.033835 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.033837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.034579 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.034581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.035377 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.035380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.035982 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.035983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.036098 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.036103 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.036162 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.036203 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260809 00:37:58.036635 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2797cd1eac417886d0fc6580e1e5f9" candidate_uuid: "af0bf2be30a0411686b22a864c772cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5" is_pre_election: true
I20260809 00:37:58.036720 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2797cd1eac417886d0fc6580e1e5f9" candidate_uuid: "af0bf2be30a0411686b22a864c772cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715" is_pre_election: true
I20260809 00:37:58.036803 16298 raft_consensus.cc:2454] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af0bf2be30a0411686b22a864c772cb8 in term 0.
I20260809 00:37:58.036806 16642 raft_consensus.cc:2454] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af0bf2be30a0411686b22a864c772cb8 in term 0.
I20260809 00:37:58.037058 16387 leader_election.cc:302] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af0bf2be30a0411686b22a864c772cb8, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.037196 16734 raft_consensus.cc:2781] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:58.037235 16734 raft_consensus.cc:482] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:58.037257 16734 raft_consensus.cc:3037] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.037964 16734 raft_consensus.cc:504] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.038121 16734 leader_election.cc:288] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [CANDIDATE]: Term 1 election: Requested vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.038291 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2797cd1eac417886d0fc6580e1e5f9" candidate_uuid: "af0bf2be30a0411686b22a864c772cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715"
I20260809 00:37:58.038326 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2797cd1eac417886d0fc6580e1e5f9" candidate_uuid: "af0bf2be30a0411686b22a864c772cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5"
I20260809 00:37:58.038388 16298 raft_consensus.cc:3037] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.038391 16642 raft_consensus.cc:3037] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.039016 16642 raft_consensus.cc:2454] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af0bf2be30a0411686b22a864c772cb8 in term 1.
I20260809 00:37:58.039099 16298 raft_consensus.cc:2454] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af0bf2be30a0411686b22a864c772cb8 in term 1.
I20260809 00:37:58.039208 16389 leader_election.cc:302] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, af0bf2be30a0411686b22a864c772cb8; no voters:
I20260809 00:37:58.039343 16734 raft_consensus.cc:2781] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:58.039546 16734 raft_consensus.cc:686] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 1 LEADER]: Becoming Leader. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Running, Role: LEADER
I20260809 00:37:58.039636 16734 consensus_queue.cc:237] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.040431 16108 catalog_manager.cc:5495] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to af0bf2be30a0411686b22a864c772cb8 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "af0bf2be30a0411686b22a864c772cb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } health_report { overall_health: UNKNOWN } } }
W20260809 00:37:58.058938 16694 tablet.cc:2369] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235877, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.126@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:58.087903 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.081044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.7.105.126:44349)
0809 00:37:58.081121 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.081124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.081266 (+ 142us) server_negotiation.cc:407] Connection header received
0809 00:37:58.081324 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.081325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.081335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.081377 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:58.081860 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.082372 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.083057 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.083236 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.083292 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.085402 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:58.085403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:58.085403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:58.085410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:58.086921 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:58.087273 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:58.087274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:58.087274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:58.087323 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:58.087489 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:58.087490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:58.087490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:58.087587 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:58.087637 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.087757 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.087830 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:37:58.088198 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.080998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44349 (local address 127.0.0.1:57898)
0809 00:37:58.081183 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.081194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.081265 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.081395 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.081396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.081724 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:58.081776 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.081778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.082445 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.082445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.082976 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.082978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.083305 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.083306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.083348 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.083857 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:58.083861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:58.085322 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:58.086990 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:58.086996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:58.086998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:58.087209 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:58.087380 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:58.087380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:58.087381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:58.087437 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:58.087641 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:58.087647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:58.087689 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.087740 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.087902 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":68,"thread_start_us":110,"threads_started":1}
I20260809 00:37:58.090315 16108 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:57898:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:37:58.090531 16108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:58.092275 16276 tablet_service.cc:1467] Processing CreateTablet for tablet aa46f9ff40da47b5a0a3ac94c9c1567d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fce12516a78046df9dbe30e19a0ad884]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:58.092275 16618 tablet_service.cc:1467] Processing CreateTablet for tablet aa46f9ff40da47b5a0a3ac94c9c1567d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fce12516a78046df9dbe30e19a0ad884]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:58.092476 16618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa46f9ff40da47b5a0a3ac94c9c1567d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.092476 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa46f9ff40da47b5a0a3ac94c9c1567d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.094177 16727 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: Bootstrap starting.
I20260809 00:37:58.094177 16728 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: Bootstrap starting.
I20260809 00:37:58.094272 16452 tablet_service.cc:1467] Processing CreateTablet for tablet aa46f9ff40da47b5a0a3ac94c9c1567d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fce12516a78046df9dbe30e19a0ad884]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:37:58.094449 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa46f9ff40da47b5a0a3ac94c9c1567d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.094591 16727 tablet_bootstrap.cc:654] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.094590 16728 tablet_bootstrap.cc:654] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.095304 16727 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: No bootstrap required, opened a new log
I20260809 00:37:58.095364 16727 ts_tablet_manager.cc:1397] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:58.095507 16727 raft_consensus.cc:348] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.095587 16727 raft_consensus.cc:374] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.095638 16727 raft_consensus.cc:729] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.095688 16727 consensus_queue.cc:260] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.095793 16727 ts_tablet_manager.cc:1428] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.095813 16738 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.095887 16738 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.096132 16728 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: No bootstrap required, opened a new log
I20260809 00:37:58.096190 16728 ts_tablet_manager.cc:1397] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:58.096200 16726 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: Bootstrap starting.
I20260809 00:37:58.096335 16728 raft_consensus.cc:348] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.096436 16728 raft_consensus.cc:374] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.096457 16728 raft_consensus.cc:729] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.096513 16728 consensus_queue.cc:260] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.096616 16726 tablet_bootstrap.cc:654] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.096637 16728 ts_tablet_manager.cc:1428] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.096714 16736 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.096850 16736 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.097440 16726 tablet_bootstrap.cc:492] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: No bootstrap required, opened a new log
I20260809 00:37:58.097508 16726 ts_tablet_manager.cc:1397] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:58.097667 16726 raft_consensus.cc:348] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.097757 16726 raft_consensus.cc:374] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.097791 16726 raft_consensus.cc:729] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.097859 16726 consensus_queue.cc:260] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.097993 16726 ts_tablet_manager.cc:1428] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.098052 16734 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.098176 16734 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.380431 16736 raft_consensus.cc:482] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:58.380541 16736 raft_consensus.cc:504] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.380879 16736 leader_election.cc:288] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.383900 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.381119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43583 (local address 127.7.105.66:35861)
0809 00:37:58.381224 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.381227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.381366 (+ 139us) server_negotiation.cc:407] Connection header received
0809 00:37:58.381409 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.381410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.381420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.381502 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:58.381663 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.382174 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383179 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.383549 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383607 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.383615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.383779 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.383829 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:37:58.383915 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.381070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35861 (local address 127.7.105.65:43583)
0809 00:37:58.381278 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.381287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.381356 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.381517 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.381518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.381520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:58.381583 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.381586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.382281 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.382282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.383087 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.383089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.383632 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.383704 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.383709 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.383761 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.383798 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":67,"threads_started":1}
I20260809 00:37:58.383983 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.381176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:35007 (local address 127.7.105.67:42359)
0809 00:37:58.381257 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.381260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.381336 (+ 76us) server_negotiation.cc:407] Connection header received
0809 00:37:58.381409 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.381410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.381420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.381472 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:37:58.381701 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.382166 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383231 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.383608 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383680 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.383688 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.383864 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.383907 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:37:58.384298 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.381063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42359 (local address 127.7.105.65:35007)
0809 00:37:58.381260 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.381271 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.381327 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.381501 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.381502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.381505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:37:58.381601 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.381603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.382258 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.382259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.383099 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.383102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.383700 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.383700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.383777 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.383782 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.383838 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.383875 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":33,"thread_start_us":75,"threads_started":1}
I20260809 00:37:58.384348 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa46f9ff40da47b5a0a3ac94c9c1567d" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8" is_pre_election: true
I20260809 00:37:58.384447 16475 raft_consensus.cc:2454] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 0.
I20260809 00:37:58.384639 16214 leader_election.cc:302] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af0bf2be30a0411686b22a864c772cb8, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.384649 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa46f9ff40da47b5a0a3ac94c9c1567d" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5" is_pre_election: true
I20260809 00:37:58.384728 16642 raft_consensus.cc:2454] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 0.
I20260809 00:37:58.384789 16736 raft_consensus.cc:2781] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:58.384858 16736 raft_consensus.cc:482] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:58.384886 16736 raft_consensus.cc:3037] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.385610 16736 raft_consensus.cc:504] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.385762 16736 leader_election.cc:288] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Requested vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.385916 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa46f9ff40da47b5a0a3ac94c9c1567d" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5"
I20260809 00:37:58.385994 16642 raft_consensus.cc:3037] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.386670 16642 raft_consensus.cc:2454] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 1.
I20260809 00:37:58.386793 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa46f9ff40da47b5a0a3ac94c9c1567d" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8"
I20260809 00:37:58.386878 16475 raft_consensus.cc:3037] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.386857 16213 leader_election.cc:302] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.386960 16736 raft_consensus.cc:2781] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:58.387123 16736 raft_consensus.cc:686] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [term 1 LEADER]: Becoming Leader. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Running, Role: LEADER
I20260809 00:37:58.387233 16736 consensus_queue.cc:237] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.387605 16779 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: TxnStatusTablet state changed. Reason: New leader ea6771fa1462499abda52ce0f673a715. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.387712 16475 raft_consensus.cc:2454] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 1.
I20260809 00:37:58.387792 16779 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:58.388080 16781 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:37:58.388034 16108 catalog_manager.cc:5495] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6771fa1462499abda52ce0f673a715 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:58.388203 16781 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:37:58.450874 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.447837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45638 (local address 127.7.105.65:35017)
0809 00:37:58.447881 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.447883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.447926 (+ 43us) server_negotiation.cc:407] Connection header received
0809 00:37:58.448046 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.448047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.448088 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.448166 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.448326 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.448889 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.450109 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.450291 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.450346 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.450560 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.450653 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.450707 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.450781 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.450797 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:58.450917 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.447783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.0.0.1:45638)
0809 00:37:58.447873 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.447886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.447939 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.448191 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.448192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.448194 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:58.448252 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.448254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.448989 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.448990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.449991 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.449993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.450369 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.450370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.450437 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.450456 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.450719 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.450729 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.450780 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.450833 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:37:58.452252 16475 raft_consensus.cc:1261] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Refusing update from remote peer ea6771fa1462499abda52ce0f673a715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.452299 16642 raft_consensus.cc:1261] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Refusing update from remote peer ea6771fa1462499abda52ce0f673a715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.452549 16736 consensus_queue.cc:1035] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.452677 16779 consensus_queue.cc:1035] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.453307 16734 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: TxnStatusTablet state changed. Reason: New leader ea6771fa1462499abda52ce0f673a715. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.453423 16734 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.453375 16738 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: TxnStatusTablet state changed. Reason: New leader ea6771fa1462499abda52ce0f673a715. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.453455 16738 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.454421 16779 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.454512 16779 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:58.454625 16786 mvcc.cc:205] Tried to move back new op lower bound from 7316422158137962496 to 7316422157874565120. Current Snapshot: MvccSnapshot[applied={T|T < 7316422158137962496}]
I20260809 00:37:58.454762 16738 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.454838 16738 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P 89627bd75a53413995bbd8036bcf3cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.455020 16788 mvcc.cc:205] Tried to move back new op lower bound from 7316422158137962496 to 7316422157874565120. Current Snapshot: MvccSnapshot[applied={T|T < 7316422158137962496 or (T in {7316422158137962496})}]
I20260809 00:37:58.454878 16736 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.455650 16736 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P ea6771fa1462499abda52ce0f673a715: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:37:58.456276 16734 tablet_replica.cc:439] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } } }
I20260809 00:37:58.456367 16734 tablet_replica.cc:442] T aa46f9ff40da47b5a0a3ac94c9c1567d P af0bf2be30a0411686b22a864c772cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:37:58.456663 16787 mvcc.cc:205] Tried to move back new op lower bound from 7316422158137962496 to 7316422157874565120. Current Snapshot: MvccSnapshot[applied={T|T < 7316422158137962496 or (T in {7316422158137962496})}]
I20260809 00:37:58.461369 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.458522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.7.105.66:35861)
0809 00:37:58.458604 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.458606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.458779 (+ 173us) server_negotiation.cc:407] Connection header received
0809 00:37:58.458836 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.458837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.458858 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.458918 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.459073 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.459525 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.460613 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.460796 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.460846 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.461039 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.461135 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.461183 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.461272 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.461290 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 00:37:58.463181 16734 consensus_queue.cc:1035] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.464818 16734 consensus_queue.cc:1035] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.470525 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.464307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37161 (local address 127.7.105.65:35017)
0809 00:37:58.464351 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.464353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.468239 (+ 3886us) server_negotiation.cc:407] Connection header received
0809 00:37:58.468301 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.468302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.468313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.468370 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.468501 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.468981 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.469817 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.470013 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.470097 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.470274 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.470315 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.470378 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.470434 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.470457 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:58.470561 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.464248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.7.105.66:37161)
0809 00:37:58.468155 (+ 3907us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.468167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.468245 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.468371 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.468372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.468373 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:58.468433 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.468435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.469094 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.469095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.469744 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.469746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.470108 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.470109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.470167 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.470188 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.470380 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.470383 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.470431 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.470476 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3865,"thread_start_us":98,"threads_started":1}
I20260809 00:37:58.481669 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.479254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:57637 (local address 127.7.105.66:35861)
0809 00:37:58.479295 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.479297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.479309 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:37:58.479377 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.479378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.479390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.479440 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.479568 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.480047 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.480884 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.481175 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.481238 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.481420 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.481455 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.481505 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.481573 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.481593 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:37:58.481706 16805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.479079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35861 (local address 127.7.105.65:57637)
0809 00:37:58.479256 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.479268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.479325 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.479442 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.479442 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.479444 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:58.479498 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.479500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.480181 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.480182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.480792 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.480793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.481237 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.481238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.481312 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.481335 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.481507 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.481509 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.481568 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.481614 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260809 00:37:58.491153 16108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.491271 16108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.491951 16108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Sending DeleteTablet for 3 replicas of tablet ac2797cd1eac417886d0fc6580e1e5f9
I20260809 00:37:58.492252 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45622
I20260809 00:37:58.492316 16452 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:53600
I20260809 00:37:58.492591 16618 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2797cd1eac417886d0fc6580e1e5f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 00:37:58.492799 16811 tablet_replica.cc:330] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: stopping tablet replica
I20260809 00:37:58.492933 16810 tablet_replica.cc:330] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: stopping tablet replica
I20260809 00:37:58.492988 16810 raft_consensus.cc:2229] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:58.493090 16810 raft_consensus.cc:2258] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.496290 16810 ts_tablet_manager.cc:1905] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.496435 16811 raft_consensus.cc:2229] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.496567 16811 raft_consensus.cc:2258] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.496902 16811 ts_tablet_manager.cc:1905] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.498131 16810 ts_tablet_manager.cc:1918] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.498210 16810 log.cc:1196] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/ac2797cd1eac417886d0fc6580e1e5f9
I20260809 00:37:58.498450 16810 ts_tablet_manager.cc:1939] T ac2797cd1eac417886d0fc6580e1e5f9 P af0bf2be30a0411686b22a864c772cb8: Deleting consensus metadata
I20260809 00:37:58.499117 16811 ts_tablet_manager.cc:1918] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.499253 16811 log.cc:1196] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/ac2797cd1eac417886d0fc6580e1e5f9
I20260809 00:37:58.499571 16811 ts_tablet_manager.cc:1939] T ac2797cd1eac417886d0fc6580e1e5f9 P 89627bd75a53413995bbd8036bcf3cf5: Deleting consensus metadata
I20260809 00:37:58.499969 16092 catalog_manager.cc:4841] TS 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359): tablet ac2797cd1eac417886d0fc6580e1e5f9 (table test-table [id=d76df6d9eb374017ada30168374c0199]) successfully deleted
I20260809 00:37:58.500204 16094 catalog_manager.cc:4841] TS af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861): tablet ac2797cd1eac417886d0fc6580e1e5f9 (table test-table [id=d76df6d9eb374017ada30168374c0199]) successfully deleted
I20260809 00:37:58.504199 16809 tablet_replica.cc:330] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: stopping tablet replica
I20260809 00:37:58.504534 16809 raft_consensus.cc:2229] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.504639 16809 raft_consensus.cc:2258] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.504930 16809 ts_tablet_manager.cc:1905] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.507083 16809 ts_tablet_manager.cc:1918] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.507213 16809 log.cc:1196] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/ac2797cd1eac417886d0fc6580e1e5f9
I20260809 00:37:58.507515 16809 ts_tablet_manager.cc:1939] T ac2797cd1eac417886d0fc6580e1e5f9 P ea6771fa1462499abda52ce0f673a715: Deleting consensus metadata
I20260809 00:37:58.507905 16091 catalog_manager.cc:4841] TS ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017): tablet ac2797cd1eac417886d0fc6580e1e5f9 (table test-table [id=d76df6d9eb374017ada30168374c0199]) successfully deleted
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:37:58.526996 16174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:58.521377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.7.105.126:44349)
0809 00:37:58.521436 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.521438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.521594 (+ 156us) server_negotiation.cc:407] Connection header received
0809 00:37:58.521676 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.521677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.521689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.521759 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:58.522347 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.522991 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.523772 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.523983 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.524092 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.526940 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57914: BlockingRecv error: recv got EOF from 127.0.0.1:57914 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM
I20260809 00:37:58.535436 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.528208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.7.105.126:44349)
0809 00:37:58.528247 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.528249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.528429 (+ 180us) server_negotiation.cc:407] Connection header received
0809 00:37:58.528500 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.528501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.528514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.528597 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:58.529049 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.529520 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.530235 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.530421 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.530498 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.532696 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:58.532698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:58.532698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:58.532706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:58.534231 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:58.534627 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:58.534628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:58.534629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:58.534687 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:58.534894 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:58.534895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:58.534896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:58.535009 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:58.535125 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.535229 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.535310 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:58.537099 16108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:58.537312 16108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:58.538720 16276 tablet_service.cc:1467] Processing CreateTablet for tablet 84b58b21e2da4e959529ada116a51e42 (DEFAULT_TABLE table=test-table [id=38ca78a28ee74590a8032ea93693e9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.538874 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b58b21e2da4e959529ada116a51e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.539989 16728 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Bootstrap starting.
I20260809 00:37:58.540161 16618 tablet_service.cc:1467] Processing CreateTablet for tablet 84b58b21e2da4e959529ada116a51e42 (DEFAULT_TABLE table=test-table [id=38ca78a28ee74590a8032ea93693e9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.540293 16618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b58b21e2da4e959529ada116a51e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.540377 16728 tablet_bootstrap.cc:654] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.541445 16727 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Bootstrap starting.
I20260809 00:37:58.541525 16452 tablet_service.cc:1467] Processing CreateTablet for tablet 84b58b21e2da4e959529ada116a51e42 (DEFAULT_TABLE table=test-table [id=38ca78a28ee74590a8032ea93693e9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.541667 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b58b21e2da4e959529ada116a51e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.541847 16727 tablet_bootstrap.cc:654] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.542635 16727 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: No bootstrap required, opened a new log
I20260809 00:37:58.542686 16727 ts_tablet_manager.cc:1397] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:58.542791 16727 raft_consensus.cc:348] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.542865 16727 raft_consensus.cc:374] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.542886 16727 raft_consensus.cc:729] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.542932 16727 consensus_queue.cc:260] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.543038 16727 ts_tablet_manager.cc:1428] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.543105 16726 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Bootstrap starting.
I20260809 00:37:58.543743 16726 tablet_bootstrap.cc:654] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.544601 16726 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: No bootstrap required, opened a new log
I20260809 00:37:58.544656 16726 ts_tablet_manager.cc:1397] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:37:58.544804 16726 raft_consensus.cc:348] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.544883 16726 raft_consensus.cc:374] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.544910 16726 raft_consensus.cc:729] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.544951 16726 consensus_queue.cc:260] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.545049 16726 ts_tablet_manager.cc:1428] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:58.545763 16728 tablet_bootstrap.cc:492] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: No bootstrap required, opened a new log
I20260809 00:37:58.545835 16728 ts_tablet_manager.cc:1397] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 00:37:58.545955 16728 raft_consensus.cc:348] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.546041 16728 raft_consensus.cc:374] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.546077 16728 raft_consensus.cc:729] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.546124 16728 consensus_queue.cc:260] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.546226 16728 ts_tablet_manager.cc:1428] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:58.567792 16736 raft_consensus.cc:482] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:58.567893 16736 raft_consensus.cc:504] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.568069 16736 leader_election.cc:288] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.568266 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b58b21e2da4e959529ada116a51e42" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8" is_pre_election: true
I20260809 00:37:58.568367 16475 raft_consensus.cc:2454] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 0.
I20260809 00:37:58.568413 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b58b21e2da4e959529ada116a51e42" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5" is_pre_election: true
I20260809 00:37:58.568539 16642 raft_consensus.cc:2454] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 0.
I20260809 00:37:58.568534 16214 leader_election.cc:302] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af0bf2be30a0411686b22a864c772cb8, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.568653 16736 raft_consensus.cc:2781] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:58.568691 16736 raft_consensus.cc:482] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:58.568718 16736 raft_consensus.cc:3037] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.569427 16736 raft_consensus.cc:504] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.569578 16736 leader_election.cc:288] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Requested vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.569710 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b58b21e2da4e959529ada116a51e42" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8"
I20260809 00:37:58.569789 16475 raft_consensus.cc:3037] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.569818 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b58b21e2da4e959529ada116a51e42" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5"
I20260809 00:37:58.569900 16642 raft_consensus.cc:3037] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.570544 16475 raft_consensus.cc:2454] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 1.
I20260809 00:37:58.570708 16642 raft_consensus.cc:2454] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 1.
I20260809 00:37:58.570745 16214 leader_election.cc:302] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af0bf2be30a0411686b22a864c772cb8, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.570835 16736 raft_consensus.cc:2781] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:58.570885 16736 raft_consensus.cc:686] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 1 LEADER]: Becoming Leader. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Running, Role: LEADER
I20260809 00:37:58.570941 16736 consensus_queue.cc:237] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.571688 16108 catalog_manager.cc:5495] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6771fa1462499abda52ce0f673a715 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "ea6771fa1462499abda52ce0f673a715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } health_report { overall_health: UNKNOWN } } }
I20260809 00:37:58.577642 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.575078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.7.105.65:35017)
0809 00:37:58.575141 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.575143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.575153 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:37:58.575200 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.575201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.575212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.575275 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.575379 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.576146 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.576873 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.577150 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.577227 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.577408 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.577442 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.577498 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.577543 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.577568 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:37:58.578549 16475 raft_consensus.cc:1261] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Refusing update from remote peer ea6771fa1462499abda52ce0f673a715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.578532 16642 raft_consensus.cc:1261] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Refusing update from remote peer ea6771fa1462499abda52ce0f673a715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.578773 16793 consensus_queue.cc:1035] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.578876 16736 consensus_queue.cc:1035] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.582381 16108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.582485 16108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.583166 16108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Sending DeleteTablet for 3 replicas of tablet 84b58b21e2da4e959529ada116a51e42
I20260809 00:37:58.583463 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b58b21e2da4e959529ada116a51e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45622
I20260809 00:37:58.583575 16809 tablet_replica.cc:330] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: stopping tablet replica
I20260809 00:37:58.583632 16809 raft_consensus.cc:2229] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:58.583726 16809 raft_consensus.cc:2258] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.584074 16452 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b58b21e2da4e959529ada116a51e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:53600
I20260809 00:37:58.584170 16810 tablet_replica.cc:330] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: stopping tablet replica
I20260809 00:37:58.584215 16810 raft_consensus.cc:2229] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.584256 16810 raft_consensus.cc:2258] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.584393 16618 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b58b21e2da4e959529ada116a51e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 00:37:58.584483 16811 tablet_replica.cc:330] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: stopping tablet replica
I20260809 00:37:58.584534 16811 raft_consensus.cc:2229] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.584589 16811 raft_consensus.cc:2258] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.584612 16809 ts_tablet_manager.cc:1905] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.585011 16811 ts_tablet_manager.cc:1905] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.586230 16108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:37:58.586393 16108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:37:58.586606 16811 ts_tablet_manager.cc:1918] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:58.586676 16811 log.cc:1196] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/84b58b21e2da4e959529ada116a51e42
I20260809 00:37:58.586894 16811 ts_tablet_manager.cc:1939] T 84b58b21e2da4e959529ada116a51e42 P 89627bd75a53413995bbd8036bcf3cf5: Deleting consensus metadata
I20260809 00:37:58.587221 16092 catalog_manager.cc:4841] TS 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359): tablet 84b58b21e2da4e959529ada116a51e42 (table test-table [id=38ca78a28ee74590a8032ea93693e9fd]) successfully deleted
I20260809 00:37:58.587919 16618 tablet_service.cc:1467] Processing CreateTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 (DEFAULT_TABLE table=test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.587925 16452 tablet_service.cc:1467] Processing CreateTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 (DEFAULT_TABLE table=test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.588027 16618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1bde68d5fb4cc799e7790f27ee4252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.588047 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1bde68d5fb4cc799e7790f27ee4252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.589166 16727 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Bootstrap starting.
I20260809 00:37:58.589418 16726 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Bootstrap starting.
I20260809 00:37:58.589514 16727 tablet_bootstrap.cc:654] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.589838 16726 tablet_bootstrap.cc:654] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.590415 16276 tablet_service.cc:1467] Processing CreateTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 (DEFAULT_TABLE table=test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:37:58.590482 16726 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: No bootstrap required, opened a new log
I20260809 00:37:58.590526 16726 ts_tablet_manager.cc:1397] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:58.590706 16726 raft_consensus.cc:348] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.590770 16726 raft_consensus.cc:374] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.590790 16726 raft_consensus.cc:729] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af0bf2be30a0411686b22a864c772cb8, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.590834 16726 consensus_queue.cc:260] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.590848 16809 ts_tablet_manager.cc:1918] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:58.590905 16809 log.cc:1196] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/84b58b21e2da4e959529ada116a51e42
I20260809 00:37:58.590921 16726 ts_tablet_manager.cc:1428] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.591137 16809 ts_tablet_manager.cc:1939] T 84b58b21e2da4e959529ada116a51e42 P ea6771fa1462499abda52ce0f673a715: Deleting consensus metadata
I20260809 00:37:58.591370 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1bde68d5fb4cc799e7790f27ee4252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:58.592677 16728 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Bootstrap starting.
I20260809 00:37:58.592794 16727 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: No bootstrap required, opened a new log
I20260809 00:37:58.592837 16727 ts_tablet_manager.cc:1397] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 00:37:58.592940 16727 raft_consensus.cc:348] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.592975 16728 tablet_bootstrap.cc:654] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:58.593000 16727 raft_consensus.cc:374] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.593021 16727 raft_consensus.cc:729] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.593061 16727 consensus_queue.cc:260] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.593143 16727 ts_tablet_manager.cc:1428] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:58.593421 16091 catalog_manager.cc:4841] TS ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017): tablet 84b58b21e2da4e959529ada116a51e42 (table test-table [id=38ca78a28ee74590a8032ea93693e9fd]) successfully deleted
I20260809 00:37:58.594097 16810 ts_tablet_manager.cc:1905] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.595636 16810 ts_tablet_manager.cc:1918] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:37:58.595703 16810 log.cc:1196] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/84b58b21e2da4e959529ada116a51e42
I20260809 00:37:58.595928 16810 ts_tablet_manager.cc:1939] T 84b58b21e2da4e959529ada116a51e42 P af0bf2be30a0411686b22a864c772cb8: Deleting consensus metadata
I20260809 00:37:58.596308 16094 catalog_manager.cc:4841] TS af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861): tablet 84b58b21e2da4e959529ada116a51e42 (table test-table [id=38ca78a28ee74590a8032ea93693e9fd]) successfully deleted
I20260809 00:37:58.596562 16728 tablet_bootstrap.cc:492] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: No bootstrap required, opened a new log
I20260809 00:37:58.596618 16728 ts_tablet_manager.cc:1397] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260809 00:37:58.596722 16728 raft_consensus.cc:348] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.596792 16728 raft_consensus.cc:374] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:58.596814 16728 raft_consensus.cc:729] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6771fa1462499abda52ce0f673a715, State: Initialized, Role: FOLLOWER
I20260809 00:37:58.596858 16728 consensus_queue.cc:260] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.596949 16728 ts_tablet_manager.cc:1428] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:58.737278 16738 raft_consensus.cc:482] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:58.737452 16738 raft_consensus.cc:504] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.737569 16736 raft_consensus.cc:482] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:37:58.737694 16738 leader_election.cc:288] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861)
I20260809 00:37:58.737681 16736 raft_consensus.cc:504] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.737846 16736 leader_election.cc:288] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.737905 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715" is_pre_election: true
I20260809 00:37:58.737949 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8" is_pre_election: true
I20260809 00:37:58.738010 16298 raft_consensus.cc:2454] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 0.
I20260809 00:37:58.738025 16475 raft_consensus.cc:2454] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 0.
I20260809 00:37:58.738207 16558 leader_election.cc:302] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.738306 16738 raft_consensus.cc:2781] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:58.738395 16738 raft_consensus.cc:482] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:58.738306 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5" is_pre_election: true
I20260809 00:37:58.738463 16738 raft_consensus.cc:3037] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.738441 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8" is_pre_election: true
I20260809 00:37:58.738508 16475 raft_consensus.cc:2454] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6771fa1462499abda52ce0f673a715 in term 0.
I20260809 00:37:58.738723 16214 leader_election.cc:302] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af0bf2be30a0411686b22a864c772cb8, ea6771fa1462499abda52ce0f673a715; no voters:
I20260809 00:37:58.738871 16736 raft_consensus.cc:2781] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:37:58.738953 16736 raft_consensus.cc:482] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:37:58.739001 16736 raft_consensus.cc:3037] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.739593 16738 raft_consensus.cc:504] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.739764 16738 leader_election.cc:288] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 election: Requested vote from peers ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017), af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861)
I20260809 00:37:58.739784 16642 raft_consensus.cc:2379] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ea6771fa1462499abda52ce0f673a715 in current term 1: Already voted for candidate 89627bd75a53413995bbd8036bcf3cf5 in this term.
I20260809 00:37:58.740020 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8"
I20260809 00:37:58.740003 16736 raft_consensus.cc:504] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.740085 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "89627bd75a53413995bbd8036bcf3cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6771fa1462499abda52ce0f673a715"
I20260809 00:37:58.740123 16475 raft_consensus.cc:3037] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:58.740176 16298 raft_consensus.cc:2379] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 89627bd75a53413995bbd8036bcf3cf5 in current term 1: Already voted for candidate ea6771fa1462499abda52ce0f673a715 in this term.
I20260809 00:37:58.740183 16736 leader_election.cc:288] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Requested vote from peers af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861), 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359)
I20260809 00:37:58.740388 16642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89627bd75a53413995bbd8036bcf3cf5"
I20260809 00:37:58.740471 16642 raft_consensus.cc:2379] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ea6771fa1462499abda52ce0f673a715 in current term 1: Already voted for candidate 89627bd75a53413995bbd8036bcf3cf5 in this term.
I20260809 00:37:58.740442 16474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1bde68d5fb4cc799e7790f27ee4252" candidate_uuid: "ea6771fa1462499abda52ce0f673a715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af0bf2be30a0411686b22a864c772cb8"
I20260809 00:37:58.740666 16214 leader_election.cc:302] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea6771fa1462499abda52ce0f673a715; no voters: 89627bd75a53413995bbd8036bcf3cf5, af0bf2be30a0411686b22a864c772cb8
I20260809 00:37:58.740764 16736 raft_consensus.cc:2726] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 00:37:58.741048 16475 raft_consensus.cc:2454] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89627bd75a53413995bbd8036bcf3cf5 in term 1.
I20260809 00:37:58.741238 16561 leader_election.cc:302] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 89627bd75a53413995bbd8036bcf3cf5, af0bf2be30a0411686b22a864c772cb8; no voters: ea6771fa1462499abda52ce0f673a715
I20260809 00:37:58.741334 16738 raft_consensus.cc:2781] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:58.741478 16738 raft_consensus.cc:686] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 LEADER]: Becoming Leader. State: Replica: 89627bd75a53413995bbd8036bcf3cf5, State: Running, Role: LEADER
I20260809 00:37:58.741562 16738 consensus_queue.cc:237] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } }
I20260809 00:37:58.742166 16108 catalog_manager.cc:5495] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "89627bd75a53413995bbd8036bcf3cf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89627bd75a53413995bbd8036bcf3cf5" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42359 } health_report { overall_health: HEALTHY } } }
I20260809 00:37:58.781162 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.778757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.7.105.67:42359)
0809 00:37:58.778817 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.778819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.778828 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:58.778859 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.778860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.778869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.778992 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.779025 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.779683 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.780522 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.780711 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.780776 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.780923 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.780955 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.781011 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.781083 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.781103 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:37:58.785954 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.783696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:53791 (local address 127.7.105.65:35017)
0809 00:37:58.783778 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.783779 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.783864 (+ 85us) server_negotiation.cc:407] Connection header received
0809 00:37:58.783902 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.783903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.783912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.783960 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.784112 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.784571 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.785326 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.785495 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.785545 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.785757 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.785783 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.785821 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.785880 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.785896 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:37:58.785992 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.783597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35017 (local address 127.7.105.67:53791)
0809 00:37:58.783787 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.783798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.783858 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.783976 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.783977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.783978 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:58.784017 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.784018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.784646 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.784647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.785251 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.785253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.785602 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.785603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.785652 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.785667 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.785842 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.785844 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.785879 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.785924 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":124,"threads_started":1}
I20260809 00:37:58.788228 16475 raft_consensus.cc:1261] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Refusing update from remote peer 89627bd75a53413995bbd8036bcf3cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.788264 16298 raft_consensus.cc:1261] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Refusing update from remote peer 89627bd75a53413995bbd8036bcf3cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:58.788442 16837 consensus_queue.cc:1035] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6771fa1462499abda52ce0f673a715" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.788561 16738 consensus_queue.cc:1035] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af0bf2be30a0411686b22a864c772cb8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:58.795692 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.793429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:50683 (local address 127.7.105.67:42359)
0809 00:37:58.793516 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:58.793518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:58.793525 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:58.793557 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:58.793558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:58.793567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:58.793620 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:58.793719 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.794156 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.794820 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.795163 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.795227 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.795436 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.795470 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.795522 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:58.795602 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:37:58.795623 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:37:58.795706 16805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:58.793365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42359 (local address 127.7.105.65:50683)
0809 00:37:58.793416 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:58.793427 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:58.793465 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:58.793623 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:58.793624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:58.793626 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:37:58.793671 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.793673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.794217 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.794218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.794749 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:58.794751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:58.795302 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:58.795303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:58.795363 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:58.795370 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:37:58.795534 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:37:58.795536 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:37:58.795601 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:37:58.795654 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:37:58.804761 16108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.804853 16108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:37:58.805446 16108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5773ead6ae42548a5ffc121e32ce7b: Sending DeleteTablet for 3 replicas of tablet bc1bde68d5fb4cc799e7790f27ee4252
I20260809 00:37:58.805723 16618 tablet_service.cc:1514] Processing DeleteTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 00:37:58.805750 16452 tablet_service.cc:1514] Processing DeleteTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:53600
I20260809 00:37:58.805830 16811 tablet_replica.cc:330] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: stopping tablet replica
I20260809 00:37:58.805866 16810 tablet_replica.cc:330] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: stopping tablet replica
I20260809 00:37:58.805891 16811 raft_consensus.cc:2229] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:37:58.805902 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet bc1bde68d5fb4cc799e7790f27ee4252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:37:58 UTC) from {username='oryx', principal='oryx/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45622
I20260809 00:37:58.805954 16810 raft_consensus.cc:2229] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.805961 16811 raft_consensus.cc:2258] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.805984 16809 tablet_replica.cc:330] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: stopping tablet replica
I20260809 00:37:58.806001 16810 raft_consensus.cc:2258] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.806037 16809 raft_consensus.cc:2229] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:37:58.806072 16809 raft_consensus.cc:2258] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:37:58.806258 16811 ts_tablet_manager.cc:1905] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.806460 16809 ts_tablet_manager.cc:1905] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.806476 16810 ts_tablet_manager.cc:1905] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:37:58.807997 16811 ts_tablet_manager.cc:1918] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.808027 16810 ts_tablet_manager.cc:1918] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.808099 16811 log.cc:1196] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/bc1bde68d5fb4cc799e7790f27ee4252
I20260809 00:37:58.808117 16810 log.cc:1196] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/bc1bde68d5fb4cc799e7790f27ee4252
I20260809 00:37:58.808135 16809 ts_tablet_manager.cc:1918] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:37:58.808187 16809 log.cc:1196] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/bc1bde68d5fb4cc799e7790f27ee4252
I20260809 00:37:58.808346 16811 ts_tablet_manager.cc:1939] T bc1bde68d5fb4cc799e7790f27ee4252 P 89627bd75a53413995bbd8036bcf3cf5: Deleting consensus metadata
I20260809 00:37:58.808351 16810 ts_tablet_manager.cc:1939] T bc1bde68d5fb4cc799e7790f27ee4252 P af0bf2be30a0411686b22a864c772cb8: Deleting consensus metadata
I20260809 00:37:58.808411 16809 ts_tablet_manager.cc:1939] T bc1bde68d5fb4cc799e7790f27ee4252 P ea6771fa1462499abda52ce0f673a715: Deleting consensus metadata
I20260809 00:37:58.808689 16094 catalog_manager.cc:4841] TS af0bf2be30a0411686b22a864c772cb8 (127.7.105.66:35861): tablet bc1bde68d5fb4cc799e7790f27ee4252 (table test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]) successfully deleted
I20260809 00:37:58.808739 16091 catalog_manager.cc:4841] TS ea6771fa1462499abda52ce0f673a715 (127.7.105.65:35017): tablet bc1bde68d5fb4cc799e7790f27ee4252 (table test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]) successfully deleted
I20260809 00:37:58.808966 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 16192
I20260809 00:37:58.811647 16092 catalog_manager.cc:4841] TS 89627bd75a53413995bbd8036bcf3cf5 (127.7.105.67:42359): tablet bc1bde68d5fb4cc799e7790f27ee4252 (table test-table [id=f70eb5f04d9845b9ab6990d58f83bf36]) successfully deleted
I20260809 00:37:58.819489 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 16367
I20260809 00:37:58.827286 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 16540
I20260809 00:37:58.836203 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 16073
2026-08-09T00:37:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2920 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16848](info): set up 2 sockets
Aug 09 00:37:58 dist-test-slave-1zqn krb5kdc[16848](info): commencing operation
krb5kdc: starting...
W20260809 00:37:59.951934 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:38:00 dist-test-slave-1zqn krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:38:00.083886 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36843
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:00.167990 16863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:00.168282 16863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:00.168319 16863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:00.169497 16863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:00.169575 16863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:00.169602 16863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:00.169633 16863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:00.169660 16863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:00.171237 16863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:00.171519 16863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:00.171754 16863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:00.177276 16873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:00.177273 16869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:00.177515 16863 server_base.cc:1034] running on GCE node
W20260809 00:38:00.177273 16870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:00.177865 16863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:00.178192 16863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:00.179415 16863 hybrid_clock.cc:648] HybridClock initialized: now 1786235880179388 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T00:38:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1342 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:38:00 dist-test-slave-1zqn krb5kdc[16878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:38:00 dist-test-slave-1zqn krb5kdc[16878](info): set up 2 sockets
Aug 09 00:38:00 dist-test-slave-1zqn krb5kdc[16878](info): commencing operation
krb5kdc: starting...
W20260809 00:38:01.287310 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:38:01.415776 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:33359
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:33359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:01.506493 16891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:01.506739 16891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:01.506779 16891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:01.507987 16891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:01.508046 16891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:01.508118 16891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:01.508140 16891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:01.508172 16891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:01.509785 16891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:33359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:01.510080 16891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:01.510314 16891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:01.515489 16900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:01.515496 16897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:01.515645 16891 server_base.cc:1034] running on GCE node
W20260809 00:38:01.515518 16896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:01.516126 16891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:01.516397 16891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:01.517578 16891 hybrid_clock.cc:648] HybridClock initialized: now 1786235881517564 us; error 33 us; skew 500 ppm
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:01.525111 16891 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:38:01.525605 16891 webserver.cc:460] Webserver started at http://127.7.105.126:45299/ using document root <none> and password file <none>
I20260809 00:38:01.525866 16891 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:01.525938 16891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:01.526098 16891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:01.527185 16891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "c897fbce822a4cb99bb498b8a1666db2"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "6eaffc1a389d032130e99387a16b0512"
server_key_iv: "d27d9d9f48025cf9a40010370eaa423f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.527576 16891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "c897fbce822a4cb99bb498b8a1666db2"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "6eaffc1a389d032130e99387a16b0512"
server_key_iv: "d27d9d9f48025cf9a40010370eaa423f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.529079 16891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:38:01.530012 16908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.530326 16891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:01.530442 16891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "c897fbce822a4cb99bb498b8a1666db2"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "6eaffc1a389d032130e99387a16b0512"
server_key_iv: "d27d9d9f48025cf9a40010370eaa423f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.530521 16891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:01.543298 16891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:01.545514 16891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:01.545691 16891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:01.550091 16891 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:33359
I20260809 00:38:01.550144 16971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:33359 every 8 connection(s)
I20260809 00:38:01.550515 16891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:38:01.551437 16972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:01.551797 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 16891
I20260809 00:38:01.551928 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:38:01.552174 7589 external_mini_cluster.cc:1428] Setting key 4485d63012b7290b1ac3b9ad8b412f38
I20260809 00:38:01.554637 16972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Bootstrap starting.
I20260809 00:38:01.555357 16972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:01.555685 16972 log.cc:824] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:01.556491 16972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: No bootstrap required, opened a new log
I20260809 00:38:01.558966 16972 raft_consensus.cc:348] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:01.559165 16972 raft_consensus.cc:374] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:01.559221 16972 raft_consensus.cc:729] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c897fbce822a4cb99bb498b8a1666db2, State: Initialized, Role: FOLLOWER
I20260809 00:38:01.559363 16972 consensus_queue.cc:260] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:01.559473 16972 raft_consensus.cc:388] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:01.559520 16972 raft_consensus.cc:482] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:38:01.559573 16972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:01.560344 16972 raft_consensus.cc:504] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:01.560491 16972 leader_election.cc:302] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c897fbce822a4cb99bb498b8a1666db2; no voters:
I20260809 00:38:01.560685 16972 leader_election.cc:288] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:38:01.560748 16979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:38:01.560952 16979 raft_consensus.cc:686] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 LEADER]: Becoming Leader. State: Replica: c897fbce822a4cb99bb498b8a1666db2, State: Running, Role: LEADER
I20260809 00:38:01.560989 16972 sys_catalog.cc:564] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:38:01.561672 16979 consensus_queue.cc:237] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:01.562112 16982 sys_catalog.cc:455] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c897fbce822a4cb99bb498b8a1666db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } } }
I20260809 00:38:01.562192 16982 sys_catalog.cc:458] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:01.562639 16981 sys_catalog.cc:455] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c897fbce822a4cb99bb498b8a1666db2. Latest consensus state: current_term: 1 leader_uuid: "c897fbce822a4cb99bb498b8a1666db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } } }
I20260809 00:38:01.563050 16981 sys_catalog.cc:458] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: This master's current role is: LEADER
W20260809 00:38:01.563310 16995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:38:01.563365 16995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:38:01.563423 16993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:38:01.563566 16993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:38:01.564414 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:01.552765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.7.105.126:33359)
0809 00:38:01.552986 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:01.552997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:01.553032 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:38:01.553588 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:01.553597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:01.553659 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:01.553815 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:01.554437 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.555802 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.556754 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.557465 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.557578 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:01.560145 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:01.560148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:01.560149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:01.560160 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:01.562396 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.562956 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.562960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.562961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.563069 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.563250 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.563251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.563251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.563588 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:01.563783 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:01.563897 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:38:01.564189 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
I20260809 00:38:01.569630 16993 catalog_manager.cc:1347] Generated new cluster ID: 1edd384efd5f4ae3b624f42565b283cf
I20260809 00:38:01.569710 16993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:38:01.580369 16993 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:38:01.580948 16993 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:38:01.592378 16993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Generated new TSK 0
I20260809 00:38:01.592655 16993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:38:01.632561 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:01.732317 17001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:01.732515 17001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:01.732538 17001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:01.733834 17001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:01.733879 17001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:01.733906 17001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:38:01.735360 17001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:01.735562 17001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:01.735749 17001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:01.741109 17008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:01.741113 17012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:01.741158 17007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:01.741629 17001 server_base.cc:1034] running on GCE node
I20260809 00:38:01.741878 17001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:01.742152 17001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:01.743330 17001 hybrid_clock.cc:648] HybridClock initialized: now 1786235881743339 us; error 57 us; skew 500 ppm
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:01.746037 17001 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:38:01.746496 17001 webserver.cc:460] Webserver started at http://127.7.105.65:45121/ using document root <none> and password file <none>
I20260809 00:38:01.746742 17001 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:01.746812 17001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:01.746968 17001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:01.748225 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "ebf6a006b768458ca534f6ceae409ba3"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7aa834dcb0e0291c7546004ee9ed1700"
server_key_iv: "1855193a2aa295979d8324ad8610e376"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.748608 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "ebf6a006b768458ca534f6ceae409ba3"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7aa834dcb0e0291c7546004ee9ed1700"
server_key_iv: "1855193a2aa295979d8324ad8610e376"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.750047 17001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:38:01.750845 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.751088 17001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:01.751201 17001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "ebf6a006b768458ca534f6ceae409ba3"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7aa834dcb0e0291c7546004ee9ed1700"
server_key_iv: "1855193a2aa295979d8324ad8610e376"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.751276 17001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:01.769490 17001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:01.772132 17001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:01.772338 17001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:01.772665 17001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:01.773038 17001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:01.773113 17001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.773180 17001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:01.773216 17001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.780020 17001 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:46115
I20260809 00:38:01.780157 17148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:46115 every 8 connection(s)
I20260809 00:38:01.780516 17001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:01.790119 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:01.781419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:51181 (local address 127.7.105.126:33359)
0809 00:38:01.781492 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:01.781494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:01.781688 (+ 194us) server_negotiation.cc:407] Connection header received
0809 00:38:01.782162 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:01.782163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:01.782173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:01.782238 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:01.783286 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.783944 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.784996 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.785247 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.785318 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:01.787588 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:01.787589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:01.787589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:01.787596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:01.788893 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.789289 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.789289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.789290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.789342 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.789569 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.789569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.789569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.789668 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:01.789717 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:01.789967 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 00:38:01.790041 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:38:01.790241 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17001
I20260809 00:38:01.790355 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:38:01.790544 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:01.781338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:51181)
0809 00:38:01.781544 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:01.781591 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:01.782080 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:01.782324 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:01.782328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:01.782946 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:01.783148 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:01.783152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.784070 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.784071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:01.784886 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:01.784889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.785333 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.785335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:01.785435 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:01.786031 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:01.786042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:01.787497 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:01.788973 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:01.788982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:01.788985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:01.789223 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:01.789399 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:01.789400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:01.789400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:01.789516 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:01.789726 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:01.789731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:01.789852 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 00:38:01.789938 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:38:01.790162 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260809 00:38:01.790558 7589 external_mini_cluster.cc:1428] Setting key 50821ef69aca03365f6c2a64c3c73d2a
I20260809 00:38:01.791139 17149 heartbeater.cc:347] Connected to a master server at 127.7.105.126:33359
I20260809 00:38:01.791234 17149 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:01.791455 17149 heartbeater.cc:511] Master 127.7.105.126:33359 requested a full tablet report, sending...
I20260809 00:38:01.791991 16930 ts_manager.cc:194] Registered new tserver with Master: ebf6a006b768458ca534f6ceae409ba3 (127.7.105.65:46115)
I20260809 00:38:01.792673 16930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:51181
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:38:01.819918 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:01.904492 17154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:01.904757 17154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:01.904800 17154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:01.906019 17154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:01.906090 17154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:01.906136 17154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:38:01.907649 17154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:01.907975 17154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:01.908311 17154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:01.913796 17162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:01.913836 17159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:01.914028 17154 server_base.cc:1034] running on GCE node
W20260809 00:38:01.914170 17160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:01.914440 17154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:01.914728 17154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:01.915899 17154 hybrid_clock.cc:648] HybridClock initialized: now 1786235881915885 us; error 47 us; skew 500 ppm
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:01.918499 17154 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:38:01.918979 17154 webserver.cc:460] Webserver started at http://127.7.105.66:44033/ using document root <none> and password file <none>
I20260809 00:38:01.919225 17154 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:01.919294 17154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:01.919445 17154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:01.920665 17154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "e78796b89e5d428abd219faed97ac8ca"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7b5a2416e87cafbf864208245d4344af"
server_key_iv: "1b3da362cd42e6284d171338f9ed025f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.921038 17154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "e78796b89e5d428abd219faed97ac8ca"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7b5a2416e87cafbf864208245d4344af"
server_key_iv: "1b3da362cd42e6284d171338f9ed025f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.922456 17154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:38:01.923233 17170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.923406 17154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:38:01.923516 17154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "e78796b89e5d428abd219faed97ac8ca"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7b5a2416e87cafbf864208245d4344af"
server_key_iv: "1b3da362cd42e6284d171338f9ed025f"
server_key_version: "encryptionkey@0"
I20260809 00:38:01.923593 17154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:01.936730 17154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:01.938894 17154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:01.939096 17154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:01.939365 17154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:01.939710 17154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:01.939767 17154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.939805 17154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:01.939863 17154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:01.945779 17154 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:36491
I20260809 00:38:01.945845 17302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:36491 every 8 connection(s)
I20260809 00:38:01.946163 17154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:38:01 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:01.956283 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17154
I20260809 00:38:01.956398 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:38:01.956601 7589 external_mini_cluster.cc:1428] Setting key 51700e3cc2568595ac68220e77696e85
I20260809 00:38:01.956897 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:01.947014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41121 (local address 127.7.105.126:33359)
0809 00:38:01.947151 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:01.947154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:01.947423 (+ 269us) server_negotiation.cc:407] Connection header received
0809 00:38:01.948050 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:01.948088 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:01.948106 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:01.948181 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:01.949346 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.949929 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.951086 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.951321 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.951416 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:01.953751 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:01.953753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:01.953753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:01.953761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:01.955258 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.955658 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.955659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.955660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.955732 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:01.956013 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:01.956014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:01.956014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:01.956206 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:01.956272 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:01.956634 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 00:38:01.956751 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:38:01.957033 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:01.946975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:41121)
0809 00:38:01.947226 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:01.947291 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:01.947920 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:01.948285 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:01.948288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:01.948986 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:01.949206 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:01.949213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.950045 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.950047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:01.950948 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:01.950951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:01.951425 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:01.951426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:01.951570 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:01.952104 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:01.952116 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:01.953643 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:01.955331 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:01.955338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:01.955339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:01.955565 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:01.955819 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:01.955820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:01.955820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:01.955923 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:01.956284 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:01.956291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:01.956470 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 00:38:01.956602 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 00:38:01.956787 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
I20260809 00:38:01.957544 17303 heartbeater.cc:347] Connected to a master server at 127.7.105.126:33359
I20260809 00:38:01.957655 17303 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:01.957855 17303 heartbeater.cc:511] Master 127.7.105.126:33359 requested a full tablet report, sending...
I20260809 00:38:01.958283 16930 ts_manager.cc:194] Registered new tserver with Master: e78796b89e5d428abd219faed97ac8ca (127.7.105.66:36491)
I20260809 00:38:01.958739 16930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:41121
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:38:01.988164 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:02.081386 17308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:02.081606 17308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:02.081630 17308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:02.082837 17308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:02.082913 17308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:02.082969 17308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:38:02.084525 17308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:02.084781 17308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:02.085055 17308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:02.090404 17318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.090510 17308 server_base.cc:1034] running on GCE node
W20260809 00:38:02.090381 17320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.090374 17317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.090821 17308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:02.091049 17308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:02.092209 17308 hybrid_clock.cc:648] HybridClock initialized: now 1786235882092185 us; error 38 us; skew 500 ppm
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:02.095083 17308 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:38:02.095646 17308 webserver.cc:460] Webserver started at http://127.7.105.67:34463/ using document root <none> and password file <none>
I20260809 00:38:02.095911 17308 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:02.095964 17308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:02.096184 17308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:02.097405 17308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "60e156010ef940338eab5c9024f35590"
format_stamp: "Formatted at 2026-08-09 00:38:02 on dist-test-slave-1zqn"
server_key: "3a6e1107511fcafdf5334e232a205dc9"
server_key_iv: "53ce2f3f431be938f75e916e785c53eb"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.097853 17308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "60e156010ef940338eab5c9024f35590"
format_stamp: "Formatted at 2026-08-09 00:38:02 on dist-test-slave-1zqn"
server_key: "3a6e1107511fcafdf5334e232a205dc9"
server_key_iv: "53ce2f3f431be938f75e916e785c53eb"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.099362 17308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:38:02.100231 17329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.100394 17308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:02.100492 17308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "60e156010ef940338eab5c9024f35590"
format_stamp: "Formatted at 2026-08-09 00:38:02 on dist-test-slave-1zqn"
server_key: "3a6e1107511fcafdf5334e232a205dc9"
server_key_iv: "53ce2f3f431be938f75e916e785c53eb"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.100555 17308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:02.116254 17308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:02.118701 17308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:02.118822 17308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:02.119011 17308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:02.119410 17308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:02.119470 17308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.119515 17308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:02.119544 17308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.125738 17308 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38019
I20260809 00:38:02.125823 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38019 every 8 connection(s)
I20260809 00:38:02.126189 17308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:02.134713 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17308
I20260809 00:38:02.134828 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:38:02.135011 7589 external_mini_cluster.cc:1428] Setting key 10443b2d7b35e0d7df196409000a77e3
I20260809 00:38:02.137054 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.127061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:33983 (local address 127.7.105.126:33359)
0809 00:38:02.127138 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.127140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.127406 (+ 266us) server_negotiation.cc:407] Connection header received
0809 00:38:02.128004 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.128005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.128019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.128416 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.129396 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.129991 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.131222 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.131453 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.131517 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.133769 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:02.133770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:02.133771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:02.133778 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:02.135507 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:02.136017 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:02.136018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:02.136019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:02.136111 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:02.136410 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:02.136412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:02.136412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:02.136559 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:02.136632 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:02.136906 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 00:38:02.136988 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:38:02.137180 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.127038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:33983)
0809 00:38:02.127274 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.127311 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.127880 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.128499 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.128503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.129092 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.129287 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.129291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.130087 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.130088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.131135 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.131138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.131534 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.131535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.131658 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.132175 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.132183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.133682 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:02.135593 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:02.135602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:02.135604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:02.135922 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:02.136189 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:02.136190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:02.136190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:02.136315 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:02.136636 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:02.136641 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:02.136761 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 00:38:02.136847 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:38:02.136973 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":142,"threads_started":1}
I20260809 00:38:02.137707 17460 heartbeater.cc:347] Connected to a master server at 127.7.105.126:33359
I20260809 00:38:02.137840 17460 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:02.138113 17460 heartbeater.cc:511] Master 127.7.105.126:33359 requested a full tablet report, sending...
I20260809 00:38:02.138522 16930 ts_manager.cc:194] Registered new tserver with Master: 60e156010ef940338eab5c9024f35590 (127.7.105.67:38019)
I20260809 00:38:02.138942 16930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:33983
I20260809 00:38:02.147372 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
Entry for principal oryx/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab.
WARNING: no policy specified for oryx/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
Entry for principal oryx/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab.
WARNING: no policy specified for oryx/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
Entry for principal oryx/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab.
I20260809 00:38:02.234992 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17001
I20260809 00:38:02.248160 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17154
I20260809 00:38:02.254878 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17308
I20260809 00:38:02.261255 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 16891
I20260809 00:38:02.269333 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:33359
--webserver_interface=127.7.105.126
--webserver_port=45299
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:33359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:02.354302 17470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:02.354564 17470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:02.354602 17470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:02.355770 17470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:02.355840 17470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:02.355867 17470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:02.355899 17470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:02.355924 17470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:02.357479 17470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:33359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=45299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:02.357769 17470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:02.358013 17470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:38:02.363292 17470 server_base.cc:1034] running on GCE node
W20260809 00:38:02.363308 17478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.363312 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.363441 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.363766 17470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:02.364224 17470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:02.365458 17470 hybrid_clock.cc:648] HybridClock initialized: now 1786235882365440 us; error 26 us; skew 500 ppm
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:02.368414 17470 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:38:02.368892 17470 webserver.cc:460] Webserver started at http://127.7.105.126:45299/ using document root <none> and password file <none>
I20260809 00:38:02.369292 17470 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:02.369423 17470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:02.370955 17470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:38:02.371809 17485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.372018 17470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:02.372159 17470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "c897fbce822a4cb99bb498b8a1666db2"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "6eaffc1a389d032130e99387a16b0512"
server_key_iv: "d27d9d9f48025cf9a40010370eaa423f"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.372579 17470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:02.386204 17470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:02.388320 17470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:02.388494 17470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:02.392597 17470 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:33359
I20260809 00:38:02.392639 17539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:33359 every 8 connection(s)
I20260809 00:38:02.393081 17470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:38:02.394703 17541 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:38:02.394860 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17470
I20260809 00:38:02.395171 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:46115
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:33359
--webserver_port=45121
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:38:02.395612 17541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Bootstrap starting.
I20260809 00:38:02.405601 17541 log.cc:824] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:02.407380 17541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:38:02.407693 17541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Bootstrap complete.
I20260809 00:38:02.410207 17541 raft_consensus.cc:348] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:02.410461 17541 raft_consensus.cc:729] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c897fbce822a4cb99bb498b8a1666db2, State: Initialized, Role: FOLLOWER
I20260809 00:38:02.410655 17541 consensus_queue.cc:260] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:02.410739 17541 raft_consensus.cc:388] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:38:02.410795 17541 raft_consensus.cc:482] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:38:02.410848 17541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:38:02.411847 17541 raft_consensus.cc:504] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:02.411968 17541 leader_election.cc:302] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c897fbce822a4cb99bb498b8a1666db2; no voters:
I20260809 00:38:02.412200 17541 leader_election.cc:288] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:38:02.412238 17545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:38:02.412376 17545 raft_consensus.cc:686] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [term 2 LEADER]: Becoming Leader. State: Replica: c897fbce822a4cb99bb498b8a1666db2, State: Running, Role: LEADER
I20260809 00:38:02.412504 17541 sys_catalog.cc:564] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:38:02.412554 17545 consensus_queue.cc:237] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } }
I20260809 00:38:02.412917 17546 sys_catalog.cc:455] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c897fbce822a4cb99bb498b8a1666db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } } }
I20260809 00:38:02.412933 17547 sys_catalog.cc:455] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c897fbce822a4cb99bb498b8a1666db2. Latest consensus state: current_term: 2 leader_uuid: "c897fbce822a4cb99bb498b8a1666db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c897fbce822a4cb99bb498b8a1666db2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 33359 } } }
I20260809 00:38:02.413017 17546 sys_catalog.cc:458] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:02.413040 17547 sys_catalog.cc:458] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:02.413249 17551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:38:02.413844 17551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:38:02.414050 17551 catalog_manager.cc:1259] Loaded cluster ID: 1edd384efd5f4ae3b624f42565b283cf
I20260809 00:38:02.414280 17551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:38:02.415099 17551 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:38:02.415282 17551 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c897fbce822a4cb99bb498b8a1666db2: Loaded TSK: 0
I20260809 00:38:02.415493 17551 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:38:02.504122 17543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:02.504333 17543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:02.504359 17543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:02.505591 17543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:02.505635 17543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:02.505663 17543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:38:02.507150 17543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.65.keytab
--principal=oryx/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:46115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=45121
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:02.507395 17543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:02.507655 17543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:02.514287 17572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.514307 17574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.514606 17543 server_base.cc:1034] running on GCE node
W20260809 00:38:02.514428 17571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.514940 17543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:02.515224 17543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:02.516384 17543 hybrid_clock.cc:648] HybridClock initialized: now 1786235882516361 us; error 38 us; skew 500 ppm
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:02.519485 17543 init.cc:377] Logged in from keytab as oryx/127.7.105.65@KRBTEST.COM (short username oryx)
I20260809 00:38:02.520135 17543 webserver.cc:460] Webserver started at http://127.7.105.65:45121/ using document root <none> and password file <none>
I20260809 00:38:02.520438 17543 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:02.520534 17543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:02.522563 17543 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:38:02.523339 17583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.523566 17543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:38:02.523675 17543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "ebf6a006b768458ca534f6ceae409ba3"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7aa834dcb0e0291c7546004ee9ed1700"
server_key_iv: "1855193a2aa295979d8324ad8610e376"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.524201 17543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:02.562166 17543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:02.565055 17543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:02.565234 17543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:02.565553 17543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:02.566033 17543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:02.566089 17543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.566125 17543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:02.566152 17543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.574419 17543 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:46115
I20260809 00:38:02.574488 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:46115 every 8 connection(s)
I20260809 00:38:02.575074 17543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:38:02.582609 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17543
I20260809 00:38:02.583005 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:36491
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:33359
--webserver_port=44033
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.585811 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.575894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:34813)
0809 00:38:02.576146 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.576198 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.576896 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.577398 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.577402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.578243 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.578535 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.578542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.579744 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.579745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.581319 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.581322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.581950 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.581951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.582116 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.582876 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.582885 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.585563 (+ 2678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":118,"threads_started":1}
W20260809 00:38:02.585970 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.576035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:34813 (local address 127.7.105.126:33359)
0809 00:38:02.576250 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.576257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.576291 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:38:02.577145 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.577151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.577195 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.577305 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.578672 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.579625 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.581435 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.581832 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.581920 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.585753 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:34813: BlockingRecv error: recv got EOF from 127.7.105.65:34813 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
W20260809 00:38:02.586395 17719 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:33359 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.105.126:33359: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.592731 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.586646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:43633)
0809 00:38:02.586748 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.586760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.586807 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.586964 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.586965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.587132 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.587192 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.587194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.588233 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.588234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.589175 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.589178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.589675 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.589676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.589765 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.590300 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.590306 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.592638 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260809 00:38:02.592861 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.586718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43633 (local address 127.7.105.126:33359)
0809 00:38:02.586757 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.586759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.586794 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:38:02.586858 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.586859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.586871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.586945 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.587272 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.588129 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.589274 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.589589 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.589664 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.592776 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:43633: BlockingRecv error: recv got EOF from 127.7.105.65:43633 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.598945 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.593275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:58787)
0809 00:38:02.593390 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.593403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.593463 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.593620 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.593621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.593776 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.593819 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.593821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.594793 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.594794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.595585 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.595588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.596093 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.596094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.596182 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.596698 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.596705 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.598872 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260809 00:38:02.599064 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.593346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:58787 (local address 127.7.105.126:33359)
0809 00:38:02.593414 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.593416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.593447 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:38:02.593527 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.593528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.593540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.593606 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.593902 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.594693 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.595682 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.595992 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.596085 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.598993 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:58787: BlockingRecv error: recv got EOF from 127.7.105.65:58787 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 00:38:02.599280 17719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:38:02.670837 17723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:02.671056 17723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:02.671078 17723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:02.672298 17723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:02.672341 17723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:02.672369 17723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:38:02.673833 17723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.66.keytab
--principal=oryx/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:36491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=44033
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:02.674085 17723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:02.674366 17723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:38:02.680038 17723 server_base.cc:1034] running on GCE node
W20260809 00:38:02.680140 17729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.680078 17730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.680295 17732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.680526 17723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:02.680828 17723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:02.682001 17723 hybrid_clock.cc:648] HybridClock initialized: now 1786235882681975 us; error 47 us; skew 500 ppm
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:02.684805 17723 init.cc:377] Logged in from keytab as oryx/127.7.105.66@KRBTEST.COM (short username oryx)
I20260809 00:38:02.685263 17723 webserver.cc:460] Webserver started at http://127.7.105.66:44033/ using document root <none> and password file <none>
I20260809 00:38:02.685508 17723 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:02.685606 17723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:02.687093 17723 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:38:02.687769 17740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.688038 17723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:02.688179 17723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "e78796b89e5d428abd219faed97ac8ca"
format_stamp: "Formatted at 2026-08-09 00:38:01 on dist-test-slave-1zqn"
server_key: "7b5a2416e87cafbf864208245d4344af"
server_key_iv: "1b3da362cd42e6284d171338f9ed025f"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.688567 17723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:02.726578 17723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:02.729238 17723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:02.729422 17723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:02.729696 17723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:02.730044 17723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:02.730106 17723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.730149 17723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:02.730173 17723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.736796 17723 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:36491
I20260809 00:38:02.736897 17864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:36491 every 8 connection(s)
I20260809 00:38:02.737257 17723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:38:02.740813 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17723
I20260809 00:38:02.741179 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:38019
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:33359
--webserver_port=34463
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.746243 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.738244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:35491)
0809 00:38:02.738429 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.738474 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.738989 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.739254 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.739258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.739875 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.740147 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.740153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.741132 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.741133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.742221 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.742224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.742831 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.742832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.742947 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.743587 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.743597 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.746075 (+ 2478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260809 00:38:02.746320 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.738281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35491 (local address 127.7.105.126:33359)
0809 00:38:02.738362 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.738364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.738577 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:38:02.739093 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.739094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.739109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.739174 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.740254 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.740983 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.742330 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.742704 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.742787 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.746226 (+ 3439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:35491: BlockingRecv error: recv got EOF from 127.7.105.66:35491 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 00:38:02.746590 17865 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:33359 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.105.126:33359: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.751328 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.746875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:44721)
0809 00:38:02.746923 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.746941 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.746986 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.747136 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.747138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.747279 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.747325 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.747327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.747974 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.747975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.748568 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.748570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.748939 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.748940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.749008 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.749472 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.749477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.751238 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:38:02.751390 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.746933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:44721 (local address 127.7.105.126:33359)
0809 00:38:02.746999 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.747001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.747009 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:38:02.747040 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.747041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.747051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.747120 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.747404 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.747887 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.748671 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.748861 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.748936 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.751321 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:44721: BlockingRecv error: recv got EOF from 127.7.105.66:44721 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.756134 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.751719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:35541)
0809 00:38:02.751761 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.751771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.751810 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.751968 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.751970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.752162 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.752198 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.752199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.752882 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.752883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.753382 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.753383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.753750 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.753751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.753804 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.754258 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.754264 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.756007 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 00:38:02.756194 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.751795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35541 (local address 127.7.105.126:33359)
0809 00:38:02.751859 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.751861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.751869 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:38:02.751891 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.751892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.751901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.751955 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.752317 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.752799 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.753475 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.753674 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.753753 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.756127 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:35541: BlockingRecv error: recv got EOF from 127.7.105.66:35541 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 00:38:02.756407 17865 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:38:02.827610 17868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:02.827838 17868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:02.827863 17868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:02.829232 17868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:02.829282 17868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:02.829311 17868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:38:02.830879 17868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/krb5kdc/oryx_127.7.105.67.keytab
--principal=oryx/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:38019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=34463
--tserver_master_addrs=127.7.105.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:02.831151 17868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:02.831400 17868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:02.837081 17876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.837062 17874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:02.837201 17873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:02.837280 17868 server_base.cc:1034] running on GCE node
I20260809 00:38:02.837568 17868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:02.837854 17868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:02.839083 17868 hybrid_clock.cc:648] HybridClock initialized: now 1786235882839054 us; error 48 us; skew 500 ppm
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:02.841883 17868 init.cc:377] Logged in from keytab as oryx/127.7.105.67@KRBTEST.COM (short username oryx)
I20260809 00:38:02.842465 17868 webserver.cc:460] Webserver started at http://127.7.105.67:34463/ using document root <none> and password file <none>
I20260809 00:38:02.842711 17868 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:02.842811 17868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:02.844480 17868 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:38:02.845183 17883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.845395 17868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:38:02.845501 17868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "60e156010ef940338eab5c9024f35590"
format_stamp: "Formatted at 2026-08-09 00:38:02 on dist-test-slave-1zqn"
server_key: "3a6e1107511fcafdf5334e232a205dc9"
server_key_iv: "53ce2f3f431be938f75e916e785c53eb"
server_key_version: "encryptionkey@0"
I20260809 00:38:02.845875 17868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:02.862031 17868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:02.864179 17868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:02.864362 17868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:02.864624 17868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:02.864949 17868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:02.865015 17868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.865061 17868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:02.865085 17868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:02.871115 17868 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38019
I20260809 00:38:02.871168 18012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38019 every 8 connection(s)
I20260809 00:38:02.871542 17868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:38:02.877894 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 17868
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:38:02.880244 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.872351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:49935 (local address 127.7.105.126:33359)
0809 00:38:02.872430 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.872432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.872689 (+ 257us) server_negotiation.cc:407] Connection header received
0809 00:38:02.873169 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.873170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.873181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.873236 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.874480 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.875036 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.876301 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.876547 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.876614 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.880195 (+ 3581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:49935: BlockingRecv error: recv got EOF from 127.7.105.67:49935 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:38:02.880309 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.872358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:49935)
0809 00:38:02.872562 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.872601 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.873082 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.873320 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.873324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.873946 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.874373 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.874381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.875128 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.875129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.876180 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.876183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.876641 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.876642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.876762 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.877323 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.877333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.880101 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260809 00:38:02.880595 18013 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:33359 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.105.126:33359: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.885452 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.880853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:45331)
0809 00:38:02.880888 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.880898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.880963 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.881088 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.881089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.881223 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.881269 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.881271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.882139 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.882140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.882641 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.882642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.883031 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.883032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.883107 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.883545 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.883550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.885362 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 00:38:02.885509 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.880902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:45331 (local address 127.7.105.126:33359)
0809 00:38:02.880964 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.880966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.880975 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:38:02.881018 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.881019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.881028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.881083 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.881358 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.881805 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.882732 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.882942 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.883030 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.885440 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:45331: BlockingRecv error: recv got EOF from 127.7.105.67:45331 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:38:02.886226 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.878518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.7.105.126:33359)
0809 00:38:02.878673 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.878676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.878685 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:38:02.878708 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.878709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.878723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.878803 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.879590 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.880382 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.881339 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.881560 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.881636 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.882797 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:02.882802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:02.882802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:02.882815 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:02.884676 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:02.885085 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:02.885089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:02.885089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:02.885197 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:02.885409 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:02.885410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:02.885410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:02.885720 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:02.885863 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:02.885998 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 00:38:02.886117 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:02 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:02.890167 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:02.886014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:60055)
0809 00:38:02.886058 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:02.886068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:02.886106 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:02.886289 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:02.886289 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:02.886438 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:02.886470 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.886471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.887247 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.887248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.887754 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:02.887755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.888147 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.888148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:02.888203 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.888532 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:02.888536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:02.890121 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:38:02.890292 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:02.886071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:60055 (local address 127.7.105.126:33359)
0809 00:38:02.886110 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:02.886112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:02.886119 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:38:02.886165 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:02.886166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:02.886177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:02.886207 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:02.886561 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.887185 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.887840 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:02.888043 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:02.888149 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:02.890219 (+ 2070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:60055: BlockingRecv error: recv got EOF from 127.7.105.67:60055 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:38:02.890304 18013 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:03.604930 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:03.599727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:43659)
0809 00:38:03.599879 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:03.599891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:03.599961 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:03.600217 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:03.600219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:03.600409 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:03.600463 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.600465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.601157 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.601158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.601980 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.601983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.602324 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.602324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.602369 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.602897 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:03.602902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:03.604845 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
W20260809 00:38:03.605130 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:03.599812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43659 (local address 127.7.105.126:33359)
0809 00:38:03.599954 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:03.599956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:03.599965 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:38:03.600036 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:03.600037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:03.600047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:03.600167 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:03.600537 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.601041 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.602049 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.602249 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.602305 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.605056 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:43659: BlockingRecv error: recv got EOF from 127.7.105.65:43659 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:03.761860 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:03.756849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:50277)
0809 00:38:03.757067 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:03.757079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:03.757162 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:03.757289 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:03.757290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:03.757472 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:03.757522 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.757523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.758183 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.758184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.758788 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.758790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.759157 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.759158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.759204 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.759798 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:03.759803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:03.761778 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":71,"threads_started":1}
W20260809 00:38:03.762046 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:03.756956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:50277 (local address 127.7.105.126:33359)
0809 00:38:03.757006 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:03.757008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:03.757160 (+ 152us) server_negotiation.cc:407] Connection header received
0809 00:38:03.757205 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:03.757206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:03.757219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:03.757270 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:03.757585 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.758096 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.758869 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.759082 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.759137 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.761887 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:50277: BlockingRecv error: recv got EOF from 127.7.105.66:50277 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:03 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:03.896557 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:03.890747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:41669)
0809 00:38:03.890957 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:03.890970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:03.891050 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:03.891234 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:03.891235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:03.891434 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:03.891491 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.891493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.892504 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.892506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.893138 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:03.893140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.893564 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.893566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:03.893637 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.894129 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:03.894135 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:03.896424 (+ 2289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
W20260809 00:38:03.896592 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:03.890850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:41669 (local address 127.7.105.126:33359)
0809 00:38:03.890901 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:03.890903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:03.891109 (+ 206us) server_negotiation.cc:407] Connection header received
0809 00:38:03.891150 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:03.891151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:03.891165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:03.891227 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:03.891602 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.892365 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.893252 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:03.893479 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:03.893566 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:03.896517 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:41669: BlockingRecv error: recv got EOF from 127.7.105.67:41669 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:04.610713 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:04.605480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:39793)
0809 00:38:04.605646 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:04.605659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:04.605721 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:04.605882 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:04.605883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:04.606065 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:04.606115 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.606116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.606831 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.606831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.607386 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.607387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.607788 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.607789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.607831 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.608340 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:04.608345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:04.610608 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":117,"threads_started":1}
W20260809 00:38:04.610777 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:04.605603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:39793 (local address 127.7.105.126:33359)
0809 00:38:04.605735 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:04.605738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:04.605754 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:38:04.605777 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:04.605778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:04.605789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:04.605855 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:04.606218 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.606752 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.607491 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.607709 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.607777 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.610704 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:39793: BlockingRecv error: recv got EOF from 127.7.105.65:39793 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:04.767804 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:04.762462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:45067)
0809 00:38:04.762640 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:04.762651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:04.762739 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:04.763023 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:04.763025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:04.763252 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:04.763305 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.763306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.764100 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.764101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.764702 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.764703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.765123 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.765125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.765190 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.765748 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:04.765759 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:04.767695 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
W20260809 00:38:04.767864 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:04.762533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:45067 (local address 127.7.105.126:33359)
0809 00:38:04.762631 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:04.762634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:04.762723 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:38:04.762851 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:04.762852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:04.762866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:04.763023 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:04.763407 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.763970 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.764796 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.765023 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.765122 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.767790 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:45067: BlockingRecv error: recv got EOF from 127.7.105.66:45067 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:04 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:04.902132 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:04.897110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:43539)
0809 00:38:04.897320 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:04.897335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:04.897406 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:04.897569 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:04.897570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:04.897769 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:04.897822 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.897824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.898541 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.898542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.899096 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:04.899098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.899477 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.899478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:04.899545 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.900084 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:04.900088 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:04.902049 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
W20260809 00:38:04.902225 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:04.897284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:43539 (local address 127.7.105.126:33359)
0809 00:38:04.897353 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:04.897355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:04.897386 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:38:04.897463 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:04.897464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:04.897481 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:04.897549 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:04.897904 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.898454 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.899188 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:04.899386 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:04.899467 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:04.902145 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:43539: BlockingRecv error: recv got EOF from 127.7.105.67:43539 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:05.616644 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:05.611299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:46795)
0809 00:38:05.611481 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:05.611492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:05.611585 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:05.611798 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:05.611799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:05.612021 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:05.612159 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.612162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.612874 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.612875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.613519 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.613522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.613898 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.613900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.613964 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.614537 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:05.614543 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:05.616541 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":124,"threads_started":1}
W20260809 00:38:05.616694 18028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:05.611402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:46795 (local address 127.7.105.126:33359)
0809 00:38:05.611522 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:05.611525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:05.611536 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:38:05.611676 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:05.611677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:05.611693 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:05.611779 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:05.612246 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.612805 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.613593 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.613790 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.613847 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.616623 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:46795: BlockingRecv error: recv got EOF from 127.7.105.65:46795 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:05.773553 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:05.768379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:48219)
0809 00:38:05.768620 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:05.768639 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:05.768720 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:05.768908 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:05.768909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:05.769107 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:05.769158 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.769160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.769847 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.769848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.770366 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.770368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.770742 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.770743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.770806 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.771297 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:05.771302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:05.773493 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":129,"threads_started":1}
W20260809 00:38:05.773687 18028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:05.768525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:48219 (local address 127.7.105.126:33359)
0809 00:38:05.768604 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:05.768606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:05.768697 (+ 91us) server_negotiation.cc:407] Connection header received
0809 00:38:05.768798 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:05.768799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:05.768813 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:05.768898 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:05.769244 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.769751 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.770449 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.770651 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.770743 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.773596 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:48219: BlockingRecv error: recv got EOF from 127.7.105.66:48219 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:05 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:05.907800 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:05.902689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:33079)
0809 00:38:05.902902 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:05.902915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:05.903007 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:05.903209 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:05.903211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:05.903401 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:05.903450 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.903452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.904223 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.904224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.904854 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:05.904857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.905273 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.905274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:05.905332 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.905834 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:05.905839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:05.907694 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":136,"threads_started":1}
W20260809 00:38:05.907859 18028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:05.902801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:33079 (local address 127.7.105.126:33359)
0809 00:38:05.902887 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:05.902889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:05.902988 (+ 99us) server_negotiation.cc:407] Connection header received
0809 00:38:05.903105 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:05.903106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:05.903120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:05.903198 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:05.903535 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.904126 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.904951 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:05.905212 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:05.905276 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:05.907791 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:33079: BlockingRecv error: recv got EOF from 127.7.105.67:33079 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:06.627607 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:06.617214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:39333)
0809 00:38:06.617416 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:06.617430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:06.617507 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:06.617696 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:06.617697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:06.617960 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:06.618030 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.618032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.618729 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.618730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.619266 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.619268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.619651 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.619652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.619710 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.625529 (+ 5819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:06.625536 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:06.627521 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
W20260809 00:38:06.627750 18034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:06.617326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:39333 (local address 127.7.105.126:33359)
0809 00:38:06.617481 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:06.617484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:06.617496 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:38:06.617592 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:06.617593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:06.617604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:06.617684 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:06.618102 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.618641 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.619357 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.619562 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.619649 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.627654 (+ 8005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:39333: BlockingRecv error: recv got EOF from 127.7.105.65:39333 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:06.779111 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:06.774085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:35193)
0809 00:38:06.774278 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:06.774289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:06.774399 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:06.774626 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:06.774627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:06.774819 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:06.774866 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.774868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.775629 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.775630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.776240 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.776242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.776645 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.776646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.776705 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.777219 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:06.777224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:06.779047 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
W20260809 00:38:06.779263 18034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:06.774160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35193 (local address 127.7.105.126:33359)
0809 00:38:06.774254 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:06.774257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:06.774376 (+ 119us) server_negotiation.cc:407] Connection header received
0809 00:38:06.774475 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:06.774475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:06.774496 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:06.774614 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:06.774951 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.775531 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.776338 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.776532 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.776631 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.779174 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:35193: BlockingRecv error: recv got EOF from 127.7.105.66:35193 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:06 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:06.913322 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:06.908357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:40343)
0809 00:38:06.908533 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:06.908557 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:06.908626 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:06.908758 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:06.908760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:06.908946 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:06.909002 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.909003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.909687 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.909688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.910246 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:06.910249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.910584 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.910585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:06.910627 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.911214 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:06.911219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:06.913223 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
W20260809 00:38:06.913426 18034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:06.908486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:40343 (local address 127.7.105.126:33359)
0809 00:38:06.908562 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:06.908564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:06.908608 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:38:06.908681 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:06.908682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:06.908694 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:06.908735 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:06.909108 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.909614 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.910324 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:06.910505 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:06.910558 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:06.913333 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:40343: BlockingRecv error: recv got EOF from 127.7.105.67:40343 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:07.633517 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:07.628287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:46417)
0809 00:38:07.628484 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:07.628520 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:07.628604 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:07.628784 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:07.628785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:07.629015 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:07.629068 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.629070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.629999 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.630001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.630556 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.630559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.630944 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.630945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.631003 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.631473 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:07.631479 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:07.633448 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
W20260809 00:38:07.633621 18039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:07.628356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:46417 (local address 127.7.105.126:33359)
0809 00:38:07.628503 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:07.628505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:07.628601 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:38:07.628660 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:07.628660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:07.628671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:07.628734 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:07.629165 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.629919 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.630643 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.630833 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.630890 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.633544 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:46417: BlockingRecv error: recv got EOF from 127.7.105.65:46417 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:07.784441 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:07.779615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:43245)
0809 00:38:07.779798 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:07.779814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:07.779887 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:07.780130 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:07.780132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:07.780314 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:07.780362 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.780364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.781078 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.781080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.781607 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.781609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.781978 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.781979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.782044 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.782514 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:07.782518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:07.784380 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
W20260809 00:38:07.784590 18039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:07.779759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:43245 (local address 127.7.105.126:33359)
0809 00:38:07.779840 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:07.779842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:07.779872 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:38:07.779969 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:07.779970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:07.779994 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:07.780082 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:07.780449 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.780983 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.781694 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.781891 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.781968 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.784496 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:43245: BlockingRecv error: recv got EOF from 127.7.105.66:43245 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:07 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:07.919312 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:07.913996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:48055)
0809 00:38:07.914187 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:07.914209 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:07.914306 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:07.914522 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:07.914523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:07.914799 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:07.914866 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.914869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.915763 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.915764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.916409 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:07.916411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.916792 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.916793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:07.916842 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.917309 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:07.917314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:07.919243 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
W20260809 00:38:07.919415 18039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:07.914148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:48055 (local address 127.7.105.126:33359)
0809 00:38:07.914208 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:07.914212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:07.914312 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:38:07.914385 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:07.914387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:07.914404 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:07.914505 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:07.914961 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.915672 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.916508 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:07.916707 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:07.916784 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:07.919345 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:48055: BlockingRecv error: recv got EOF from 127.7.105.67:48055 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:08.639293 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:08.634220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:41081)
0809 00:38:08.634390 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:08.634403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:08.634460 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:08.634631 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:08.634632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:08.634824 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:08.634879 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.634881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.635644 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.635645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.636315 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.636318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.636686 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.636686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.636728 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.637209 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:08.637214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:08.639232 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
W20260809 00:38:08.639408 18044 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:08.634290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:41081 (local address 127.7.105.126:33359)
0809 00:38:08.634449 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:08.634451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:08.634461 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:38:08.634531 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:08.634532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:08.634543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:08.634608 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:08.635026 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.635552 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.636403 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.636614 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.636675 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.639328 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:41081: BlockingRecv error: recv got EOF from 127.7.105.65:41081 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:08.789634 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:08.784899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:54929)
0809 00:38:08.785182 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:08.785193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:08.785252 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:08.785386 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:08.785387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:08.785570 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:08.785621 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.785622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.786297 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.786297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.786860 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.786862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.787221 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.787222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.787261 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.787695 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:08.787700 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:08.789533 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":224,"threads_started":1}
W20260809 00:38:08.789677 18044 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:08.784989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:54929 (local address 127.7.105.126:33359)
0809 00:38:08.785099 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:08.785101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:08.785296 (+ 195us) server_negotiation.cc:407] Connection header received
0809 00:38:08.785325 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:08.785326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:08.785338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:08.785379 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:08.785684 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.786226 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.786933 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.787153 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.787206 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.789612 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:54929: BlockingRecv error: recv got EOF from 127.7.105.66:54929 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:08 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:08.924666 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:08.919842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:49483)
0809 00:38:08.920014 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:08.920026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:08.920170 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:08.920318 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:08.920319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:08.920509 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:08.920560 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.920562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.921260 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.921261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.921798 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:08.921799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.922156 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.922157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:08.922219 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.922698 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:08.922704 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:08.924566 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
W20260809 00:38:08.924729 18044 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:08.919927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:49483 (local address 127.7.105.126:33359)
0809 00:38:08.920001 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:08.920004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:08.920150 (+ 146us) server_negotiation.cc:407] Connection header received
0809 00:38:08.920226 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:08.920227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:08.920247 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:08.920311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:08.920651 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.921164 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.921884 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:08.922080 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:08.922158 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:08.924663 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:49483: BlockingRecv error: recv got EOF from 127.7.105.67:49483 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:09.644742 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:09.639897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:39247)
0809 00:38:09.640103 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:09.640115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:09.640171 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:09.640398 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:09.640399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:09.640594 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:09.640645 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.640646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.641350 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.641351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.641895 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.641898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.642297 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.642298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.642337 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.642817 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:09.642821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:09.644682 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
W20260809 00:38:09.644909 18048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:09.639993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:39247 (local address 127.7.105.126:33359)
0809 00:38:09.640210 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:09.640212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:09.640223 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:38:09.640252 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:09.640253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:09.640266 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:09.640346 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:09.640725 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.641251 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.641994 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.642222 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.642284 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.644810 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:39247: BlockingRecv error: recv got EOF from 127.7.105.65:39247 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":137,"threads_started":1}
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:09.795459 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:09.790159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:37357)
0809 00:38:09.790348 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:09.790364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:09.790479 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:09.790670 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:09.790670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:09.790872 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:09.790932 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.790934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.791781 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.791782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.792445 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.792447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.792871 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.792872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.792936 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.793416 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:09.793421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:09.795388 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
W20260809 00:38:09.795576 18048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:09.790258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37357 (local address 127.7.105.126:33359)
0809 00:38:09.790355 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:09.790358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:09.790454 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:38:09.790558 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:09.790559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:09.790575 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:09.790651 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:09.791031 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.791675 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.792544 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.792771 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.792870 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.795498 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:37357: BlockingRecv error: recv got EOF from 127.7.105.66:37357 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:09 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:09.930595 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:09.925198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:36301)
0809 00:38:09.925411 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:09.925424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:09.925509 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:09.925674 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:09.925675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:09.925881 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:09.925998 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.926000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.926709 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.926711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.927355 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:09.927357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.927767 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.927768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:09.927816 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.928427 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:09.928432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:09.930529 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":121,"threads_started":1}
W20260809 00:38:09.930670 18048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:09.925275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36301 (local address 127.7.105.126:33359)
0809 00:38:09.925352 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:09.925354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:09.925504 (+ 150us) server_negotiation.cc:407] Connection header received
0809 00:38:09.925584 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:09.925585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:09.925599 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:09.925651 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:09.926082 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.926608 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.927494 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:09.927683 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:09.927750 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:09.930619 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:36301: BlockingRecv error: recv got EOF from 127.7.105.67:36301 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:10.651228 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:10.645330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:37865)
0809 00:38:10.645480 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:10.645501 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:10.645556 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:10.645913 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:10.645915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:10.646277 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:10.646347 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.646351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.647122 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.647124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.647734 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.647736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.648232 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.648233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.648304 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.648958 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:10.648963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:10.651107 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
W20260809 00:38:10.651302 18053 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:10.645553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:37865 (local address 127.7.105.126:33359)
0809 00:38:10.645725 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:10.645728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:10.645736 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:38:10.645762 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:10.645763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:10.645774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:10.645898 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:10.646451 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.647016 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.647832 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.648136 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.648233 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.651224 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:37865: BlockingRecv error: recv got EOF from 127.7.105.65:37865 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:10.801615 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:10.796133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:38563)
0809 00:38:10.796361 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:10.796373 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:10.796475 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:10.796695 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:10.796696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:10.796882 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:10.796931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.796933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.797835 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.797836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.798436 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.798438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.798880 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.798881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.798948 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.799458 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:10.799463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:10.801494 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":109,"threads_started":1}
W20260809 00:38:10.801647 18053 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:10.796241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:38563 (local address 127.7.105.126:33359)
0809 00:38:10.796350 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:10.796352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:10.796456 (+ 104us) server_negotiation.cc:407] Connection header received
0809 00:38:10.796601 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:10.796602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:10.796620 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:10.796687 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:10.797066 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.797746 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.798542 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.798804 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.798882 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.801587 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:38563: BlockingRecv error: recv got EOF from 127.7.105.66:38563 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:10 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:10.936311 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:10.931270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:55011)
0809 00:38:10.931416 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:10.931428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:10.931526 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:10.931676 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:10.931678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:10.931871 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:10.931921 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.931922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.932702 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.932703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.933248 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:10.933250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.933614 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.933615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:10.933674 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.934157 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:10.934161 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:10.936199 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
W20260809 00:38:10.936367 18053 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:10.931333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:55011 (local address 127.7.105.126:33359)
0809 00:38:10.931390 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:10.931392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:10.931503 (+ 111us) server_negotiation.cc:407] Connection header received
0809 00:38:10.931582 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:10.931583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:10.931604 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:10.931669 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:10.932103 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.932616 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.933338 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:10.933539 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:10.933617 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:10.936295 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:55011: BlockingRecv error: recv got EOF from 127.7.105.67:55011 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:11.661823 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:11.651748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:52485)
0809 00:38:11.651922 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:11.651936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:11.652022 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:11.652227 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:11.652227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:11.652409 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:11.652481 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.652483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.653228 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.653230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.653879 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.653881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.654293 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.654294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.654349 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.659362 (+ 5013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:11.659377 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:11.661760 (+ 2383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
W20260809 00:38:11.662051 18058 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:11.651857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:52485 (local address 127.7.105.126:33359)
0809 00:38:11.651971 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:11.651972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:11.652000 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:38:11.652127 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:11.652128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:11.652146 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:11.652205 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:11.652577 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.653140 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.653974 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.654194 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.654255 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.661967 (+ 7712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:52485: BlockingRecv error: recv got EOF from 127.7.105.65:52485 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:11.807314 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:11.802186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:43989)
0809 00:38:11.802360 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:11.802378 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:11.802475 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:11.802655 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:11.802656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:11.802839 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:11.802921 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.802924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.803752 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.803753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.804388 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.804390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.804764 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.804765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.804808 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.805262 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:11.805267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:11.807235 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
W20260809 00:38:11.807440 18058 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:11.802324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:43989 (local address 127.7.105.126:33359)
0809 00:38:11.802428 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:11.802431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:11.802465 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:38:11.802540 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:11.802541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:11.802557 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:11.802629 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:11.803021 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.803663 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.804484 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.804694 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.804756 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.807357 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:43989: BlockingRecv error: recv got EOF from 127.7.105.66:43989 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:11 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:11.942222 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:11.936841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:36065)
0809 00:38:11.937035 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:11.937049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:11.937145 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:11.937316 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:11.937317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:11.937518 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:11.937569 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.937571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.938306 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.938307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.938870 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:11.938872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.939305 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.939306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:11.939372 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.939984 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:11.939990 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:11.942157 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
W20260809 00:38:11.942418 18058 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:11.936924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36065 (local address 127.7.105.126:33359)
0809 00:38:11.937050 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:11.937052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:11.937123 (+ 71us) server_negotiation.cc:407] Connection header received
0809 00:38:11.937218 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:11.937218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:11.937233 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:11.937309 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:11.937654 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.938211 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.939024 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:11.939220 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:11.939295 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:11.942301 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:36065: BlockingRecv error: recv got EOF from 127.7.105.67:36065 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:12.668033 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:12.662470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:48435)
0809 00:38:12.662644 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:12.662680 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:12.662751 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:12.662994 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:12.662995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:12.663218 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:12.663270 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.663272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.664214 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.664215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.664838 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.664840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.665265 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.665266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.665319 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.665914 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:12.665919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:12.667961 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
W20260809 00:38:12.668309 18062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:12.662610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:48435 (local address 127.7.105.126:33359)
0809 00:38:12.662792 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:12.662793 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:12.662805 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:38:12.662848 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:12.662849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:12.662865 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:12.662974 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:12.663403 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.664124 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.664953 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.665179 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.665246 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.668224 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:48435: BlockingRecv error: recv got EOF from 127.7.105.65:48435 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:12.812763 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:12.807875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:58101)
0809 00:38:12.808043 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:12.808071 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:12.808122 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:12.808248 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:12.808249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:12.808432 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:12.808480 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.808482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.809159 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.809161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.809794 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.809797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.810150 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.810151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.810198 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.810654 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:12.810660 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:12.812701 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
W20260809 00:38:12.812958 18062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:12.807953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:58101 (local address 127.7.105.126:33359)
0809 00:38:12.808046 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:12.808048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:12.808118 (+ 70us) server_negotiation.cc:407] Connection header received
0809 00:38:12.808184 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:12.808185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:12.808196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:12.808238 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:12.808541 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.809059 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.809881 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.810077 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.810130 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.812853 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:58101: BlockingRecv error: recv got EOF from 127.7.105.66:58101 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:12 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:12.947808 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:12.942835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:52179)
0809 00:38:12.942991 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:12.943015 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:12.943072 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:12.943320 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:12.943322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:12.943520 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:12.943572 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.943573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.944364 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.944365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.944890 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:12.944891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.945274 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.945275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:12.945317 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.945798 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:12.945803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:12.947744 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
W20260809 00:38:12.947937 18062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:12.943047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:52179 (local address 127.7.105.126:33359)
0809 00:38:12.943139 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:12.943141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:12.943151 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:38:12.943175 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:12.943176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:12.943188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:12.943268 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:12.943688 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.944279 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.944972 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:12.945204 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:12.945262 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:12.947856 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:52179: BlockingRecv error: recv got EOF from 127.7.105.67:52179 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:13.673972 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:13.668665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:52269)
0809 00:38:13.668853 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:13.668865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:13.669012 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:13.669214 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:13.669215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:13.669395 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:13.669446 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.669448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.670287 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.670288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.670844 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.670845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.671249 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.671250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.671295 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.671788 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:13.671792 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:13.673896 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
W20260809 00:38:13.674106 18066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:13.668755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:52269 (local address 127.7.105.126:33359)
0809 00:38:13.668893 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:13.668901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:13.669003 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:38:13.669085 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:13.669086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:13.669097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:13.669153 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:13.669561 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.670199 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.670943 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.671171 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.671241 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.673995 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:52269: BlockingRecv error: recv got EOF from 127.7.105.65:52269 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:13.818538 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:13.813472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:50329)
0809 00:38:13.813670 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:13.813684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:13.813760 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:13.813952 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:13.813954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:13.814144 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:13.814206 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.814207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.814967 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.814968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.815543 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.815545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.815896 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.815897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.815972 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.816463 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:13.816468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:13.818433 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
W20260809 00:38:13.818697 18066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:13.813524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:50329 (local address 127.7.105.126:33359)
0809 00:38:13.813597 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:13.813600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:13.813761 (+ 161us) server_negotiation.cc:407] Connection header received
0809 00:38:13.813823 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:13.813823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:13.813835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:13.813891 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:13.814283 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.814877 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.815619 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.815804 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.815857 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.818603 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:50329: BlockingRecv error: recv got EOF from 127.7.105.66:50329 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:13 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:13.953979 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:13.948384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:60625)
0809 00:38:13.948585 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:13.948599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:13.948677 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:13.948865 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:13.948867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:13.949053 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:13.949100 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.949102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.950012 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.950013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.950579 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:13.950581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.950923 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.950924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:13.951028 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.951595 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:13.951600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:13.953901 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
W20260809 00:38:13.954138 18066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:13.948518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:60625 (local address 127.7.105.126:33359)
0809 00:38:13.948591 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:13.948594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:13.948699 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:38:13.948733 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:13.948734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:13.948751 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:13.948821 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:13.949205 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.949880 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.950663 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:13.950848 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:13.950908 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:13.954054 (+ 3146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:60625: BlockingRecv error: recv got EOF from 127.7.105.67:60625 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:14.679889 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:14.674543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:43943)
0809 00:38:14.674719 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:14.674733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:14.674821 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:14.675015 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:14.675017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:14.675245 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:14.675295 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.675297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.676159 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.676165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.676722 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.676724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.677128 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.677129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.677179 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.677661 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:14.677666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:14.679817 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
W20260809 00:38:14.680143 18070 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:14.674629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43943 (local address 127.7.105.126:33359)
0809 00:38:14.674806 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:14.674808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:14.674820 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:38:14.674870 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:14.674870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:14.674881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:14.674950 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:14.675378 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.676009 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.676830 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.677044 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.677104 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.679956 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:43943: BlockingRecv error: recv got EOF from 127.7.105.65:43943 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:14.824185 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:14.819045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:34663)
0809 00:38:14.819197 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:14.819207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:14.819261 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:14.819455 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:14.819457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:14.819649 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:14.819711 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.819713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.820378 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.820379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.820961 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.820964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.821358 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.821359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.821412 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.821981 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:14.821986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:14.824115 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260809 00:38:14.824323 18070 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:14.819154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:34663 (local address 127.7.105.126:33359)
0809 00:38:14.819226 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:14.819228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:14.819251 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:38:14.819316 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:14.819317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:14.819328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:14.819382 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:14.819788 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.820304 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.821059 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.821253 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.821311 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.824237 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:34663: BlockingRecv error: recv got EOF from 127.7.105.66:34663 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:14 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:14.959517 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:14.954497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:40879)
0809 00:38:14.954675 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:14.954686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:14.954752 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:14.954894 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:14.954895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:14.955077 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:14.955126 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.955135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.955794 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.955795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.956462 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:14.956465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.956830 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.956831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:14.956878 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.957484 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:14.957489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:14.959453 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
W20260809 00:38:14.959648 18070 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:14.954581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:40879 (local address 127.7.105.126:33359)
0809 00:38:14.954651 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:14.954653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:14.954737 (+ 84us) server_negotiation.cc:407] Connection header received
0809 00:38:14.954807 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:14.954808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:14.954821 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:14.954869 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:14.955204 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.955703 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.956546 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:14.956754 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:14.956812 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:14.959563 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:40879: BlockingRecv error: recv got EOF from 127.7.105.67:40879 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:15.690271 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:15.680541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:33705)
0809 00:38:15.680717 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:15.680730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:15.680792 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:15.681022 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:15.681023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:15.681276 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:15.681325 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.681327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.682163 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.682164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.682745 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.682747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.683251 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.683252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.683301 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.688016 (+ 4715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:15.688022 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:15.690190 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260809 00:38:15.690603 18074 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:15.680674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:33705 (local address 127.7.105.126:33359)
0809 00:38:15.680831 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:15.680833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:15.680841 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:38:15.680866 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:15.680866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:15.680886 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:15.680982 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:15.681414 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.681918 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.682916 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.683165 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.683231 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.690385 (+ 7154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:33705: BlockingRecv error: recv got EOF from 127.7.105.65:33705 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:15.830013 18075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:15.824721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:33235)
0809 00:38:15.824880 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:15.824902 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:15.824985 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:15.825161 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:15.825162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:15.825356 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:15.825409 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.825411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.826155 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.826156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.826694 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.826696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.827076 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.827077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.827142 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.827638 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:15.827643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:15.829945 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
W20260809 00:38:15.830163 18074 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:15.824837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:33235 (local address 127.7.105.126:33359)
0809 00:38:15.824904 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:15.824906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:15.824966 (+ 60us) server_negotiation.cc:407] Connection header received
0809 00:38:15.825063 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:15.825065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:15.825079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:15.825154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:15.825494 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.826054 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.826779 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.826989 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.827065 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.830092 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:33235: BlockingRecv error: recv got EOF from 127.7.105.66:33235 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:15 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:15.966418 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:15.960218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:58809)
0809 00:38:15.960413 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:15.960425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:15.960493 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:15.960633 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:15.960634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:15.960806 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:15.960854 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.960855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.961533 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.961534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.962152 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:15.962154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.962518 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.962518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:15.962560 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.963098 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:15.963103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:15.966302 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
W20260809 00:38:15.966542 18074 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:15.960303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:58809 (local address 127.7.105.126:33359)
0809 00:38:15.960375 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:15.960377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:15.960493 (+ 116us) server_negotiation.cc:407] Connection header received
0809 00:38:15.960550 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:15.960551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:15.960564 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:15.960606 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:15.960933 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.961456 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.962241 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:15.962447 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:15.962505 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:15.966452 (+ 3947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:58809: BlockingRecv error: recv got EOF from 127.7.105.67:58809 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:16.699165 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:16.690847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:59617)
0809 00:38:16.691063 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:16.691077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:16.691149 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:16.691329 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:16.691330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:16.691584 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:16.691657 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.691660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.692503 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.692504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.693180 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.693183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.693583 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.693584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.693650 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.696824 (+ 3174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:16.696830 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:16.699049 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":110,"threads_started":1}
W20260809 00:38:16.699239 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:16.691008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:59617 (local address 127.7.105.126:33359)
0809 00:38:16.691154 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:16.691161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:16.691205 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:38:16.691229 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:16.691229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:16.691240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:16.691307 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:16.691756 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.692414 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.693269 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.693486 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.693548 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.699154 (+ 5606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:59617: BlockingRecv error: recv got EOF from 127.7.105.65:59617 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:16.835870 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:16.830680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:46431)
0809 00:38:16.830880 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:16.830893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:16.830987 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:16.831166 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:16.831167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:16.831368 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:16.831416 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.831418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.832154 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.832156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.832752 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.832754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.833145 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.833146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.833206 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.833719 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:16.833724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:16.835803 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
W20260809 00:38:16.836119 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:16.830784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:46431 (local address 127.7.105.126:33359)
0809 00:38:16.830863 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:16.830866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:16.830968 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:38:16.831069 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:16.831070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:16.831091 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:16.831158 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:16.831498 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.832010 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.832841 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.833050 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.833142 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.835931 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:46431: BlockingRecv error: recv got EOF from 127.7.105.66:46431 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:16 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.67@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:16.972748 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:16.966954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.67:56445)
0809 00:38:16.967226 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:16.967249 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:16.967355 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:16.967551 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:16.967552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:16.967760 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:16.967824 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.967826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.968749 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.968750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.969343 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:16.969345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.969833 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.969834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:16.969910 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.970503 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:16.970508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:16.972686 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":101,"threads_started":1}
W20260809 00:38:16.972913 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:16.967052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:56445 (local address 127.7.105.126:33359)
0809 00:38:16.967184 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:16.967187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:16.967339 (+ 152us) server_negotiation.cc:407] Connection header received
0809 00:38:16.967425 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:16.967427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:16.967444 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:16.967523 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:16.967920 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.968647 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.969441 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:16.969768 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:16.969837 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:16.972811 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.67:56445: BlockingRecv error: recv got EOF from 127.7.105.67:56445 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 00:38:17 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:17 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.65@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:17.705027 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:17.699841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.65:58033)
0809 00:38:17.699998 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:17.700012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:17.700114 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:17.700288 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:17.700289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:17.700455 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:17.700506 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:17.700507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.701192 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.701193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:17.701734 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:17.701736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.702255 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.702256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:17.702319 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:17.702879 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:17.702890 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:17.704962 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
W20260809 00:38:17.705216 18083 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:17.699944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:58033 (local address 127.7.105.126:33359)
0809 00:38:17.700147 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:17.700150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:17.700167 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:38:17.700192 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:17.700193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:17.700205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:17.700271 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:17.700600 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.701121 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.701815 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.702133 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.702235 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:17.705143 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.65:58033: BlockingRecv error: recv got EOF from 127.7.105.65:58033 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
Aug 09 00:38:17 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:38:17 dist-test-slave-1zqn krb5kdc[16878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.105.66@KRBTEST.COM for oryx/127.7.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:38:17.841882 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:17.836495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:33359 (local address 127.7.105.66:38099)
0809 00:38:17.836805 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:17.836819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:17.836916 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:17.837199 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:17.837200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:17.837395 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:17.837444 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:17.837446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.838130 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.838131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:17.838717 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:17.838719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.839097 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.839098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:17.839156 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:17.839693 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:17.839699 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:17.841761 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.105.126:33359: Server oryx/127.7.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":91,"threads_started":1}
W20260809 00:38:17.842051 18083 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:17.836606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:38099 (local address 127.7.105.126:33359)
0809 00:38:17.836787 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:17.836790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:17.836895 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:38:17.837091 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:17.837092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:17.837113 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:17.837175 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:17.837530 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.838042 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.838815 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:17.839022 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:17.839099 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:17.841876 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.105.66:38099: BlockingRecv error: recv got EOF from 127.7.105.66:38099 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
I20260809 00:38:17.888650 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17543
I20260809 00:38:17.896021 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17723
I20260809 00:38:17.902796 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17868
I20260809 00:38:17.910178 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 17470
2026-08-09T00:38:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17731 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T00:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:17Z Disabled control of system clock
I20260809 00:38:17.943125 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34927
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:34927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:38:18.029172 18088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.029426 18088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.029452 18088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.030673 18088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:18.030726 18088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.030741 18088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.030755 18088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:18.030817 18088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:18.032243 18088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:34927
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.032544 18088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.032899 18088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.039016 18094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.039049 18088 server_base.cc:1034] running on GCE node
W20260809 00:38:18.039180 18093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.039049 18097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.039491 18088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.039765 18088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.040912 18088 hybrid_clock.cc:648] HybridClock initialized: now 1786235898040880 us; error 42 us; skew 500 ppm
I20260809 00:38:18.042097 18088 webserver.cc:460] Webserver started at http://127.7.105.126:39097/ using document root <none> and password file <none>
I20260809 00:38:18.042315 18088 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.042357 18088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.042451 18088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.043519 18088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "a022c721e0bc486c97e434dc8fdbef5b"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "81bd3be48c37eb095f96a9bea988a55c"
server_key_iv: "1c13126b11c1591361bc0f4b6a233ad1"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.043831 18088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "a022c721e0bc486c97e434dc8fdbef5b"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "81bd3be48c37eb095f96a9bea988a55c"
server_key_iv: "1c13126b11c1591361bc0f4b6a233ad1"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.045265 18088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:38:18.046136 18103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.046361 18088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:18.046438 18088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a022c721e0bc486c97e434dc8fdbef5b"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "81bd3be48c37eb095f96a9bea988a55c"
server_key_iv: "1c13126b11c1591361bc0f4b6a233ad1"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.046494 18088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.061210 18088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.061514 18088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.061632 18088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.066114 18088 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:34927
I20260809 00:38:18.066147 18164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:34927 every 8 connection(s)
I20260809 00:38:18.066532 18088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:38:18.067368 18165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:18.068291 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18088
I20260809 00:38:18.068389 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:38:18.068576 7589 external_mini_cluster.cc:1428] Setting key ab9711cea61dc12375bc839483a28f76
I20260809 00:38:18.070935 18165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b: Bootstrap starting.
I20260809 00:38:18.071673 18165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:18.072005 18165 log.cc:824] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:18.072870 18165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b: No bootstrap required, opened a new log
I20260809 00:38:18.074795 18165 raft_consensus.cc:348] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } }
I20260809 00:38:18.074971 18165 raft_consensus.cc:374] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:18.075021 18165 raft_consensus.cc:729] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a022c721e0bc486c97e434dc8fdbef5b, State: Initialized, Role: FOLLOWER
I20260809 00:38:18.075210 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.069209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.7.105.126:34927)
0809 00:38:18.069460 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.069468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.069519 (+ 51us) server_negotiation.cc:407] Connection header received
0809 00:38:18.069998 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.070006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.070072 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.070192 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.070497 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.071437 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.072626 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.073121 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.073232 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.074044 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.074048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.074048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.074238 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.074273 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.074285 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.074339 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.074449 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.074642 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 00:38:18.075186 18165 consensus_queue.cc:260] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } }
I20260809 00:38:18.075282 18165 raft_consensus.cc:388] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:38:18.075323 18165 raft_consensus.cc:482] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:38:18.075369 18165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:18.076112 18165 raft_consensus.cc:504] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } }
I20260809 00:38:18.076241 18165 leader_election.cc:302] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a022c721e0bc486c97e434dc8fdbef5b; no voters:
I20260809 00:38:18.076411 18165 leader_election.cc:288] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:38:18.076489 18171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:38:18.076673 18171 raft_consensus.cc:686] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [term 1 LEADER]: Becoming Leader. State: Replica: a022c721e0bc486c97e434dc8fdbef5b, State: Running, Role: LEADER
I20260809 00:38:18.076704 18165 sys_catalog.cc:564] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:38:18.076767 18171 consensus_queue.cc:237] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } }
I20260809 00:38:18.077239 18174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a022c721e0bc486c97e434dc8fdbef5b. Latest consensus state: current_term: 1 leader_uuid: "a022c721e0bc486c97e434dc8fdbef5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } } }
I20260809 00:38:18.077330 18174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:18.077229 18172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a022c721e0bc486c97e434dc8fdbef5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a022c721e0bc486c97e434dc8fdbef5b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34927 } } }
I20260809 00:38:18.077538 18172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:18.077692 18182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:38:18.078388 18182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:38:18.080113 18182 catalog_manager.cc:1347] Generated new cluster ID: 7bbb663d72994f66b0528fe3ef4ca12f
I20260809 00:38:18.080174 18182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:38:18.093819 18182 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:38:18.094417 18182 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:38:18.106765 18182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a022c721e0bc486c97e434dc8fdbef5b: Generated new TSK 0
I20260809 00:38:18.107029 18182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:38:18.109488 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--builtin_ntp_servers=127.7.105.84:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:38:18.196682 18192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.196888 18192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.196911 18192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.198093 18192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.198134 18192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.198159 18192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:38:18.199541 18192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.199793 18192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.200161 18192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.205590 18198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.205628 18197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.205829 18192 server_base.cc:1034] running on GCE node
W20260809 00:38:18.205588 18204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.206153 18192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.206365 18192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.207515 18192 hybrid_clock.cc:648] HybridClock initialized: now 1786235898207501 us; error 33 us; skew 500 ppm
I20260809 00:38:18.208933 18192 webserver.cc:460] Webserver started at http://127.7.105.65:33029/ using document root <none> and password file <none>
I20260809 00:38:18.209219 18192 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.209270 18192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.209465 18192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.210557 18192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "e2e060a2320e4e3a8b551c2e429a8a84"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5de80aa38b55937803ca44ea04601cb6"
server_key_iv: "ebced66bad96dbbcc36bd9f3854c84a5"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.210925 18192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "e2e060a2320e4e3a8b551c2e429a8a84"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5de80aa38b55937803ca44ea04601cb6"
server_key_iv: "ebced66bad96dbbcc36bd9f3854c84a5"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.212392 18192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:38:18.213274 18211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.213531 18192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:18.213630 18192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "e2e060a2320e4e3a8b551c2e429a8a84"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5de80aa38b55937803ca44ea04601cb6"
server_key_iv: "ebced66bad96dbbcc36bd9f3854c84a5"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.213706 18192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.228361 18192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.228694 18192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.228863 18192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.229097 18192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:18.229458 18192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:18.229496 18192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.229519 18192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:18.229532 18192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.235749 18192 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:43953
I20260809 00:38:18.235803 18334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:43953 every 8 connection(s)
I20260809 00:38:18.236155 18192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:38:18.241688 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.236943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:48677 (local address 127.7.105.126:34927)
0809 00:38:18.237021 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.237023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.237282 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:38:18.237758 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.237759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.237769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.237838 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.238520 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.239073 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.240045 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.240304 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.240364 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.241292 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.241294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.241294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.241299 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.241317 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.241322 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.241394 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.241540 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.241578 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:38:18.241895 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.236941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34927 (local address 127.7.105.65:48677)
0809 00:38:18.237149 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:18.237191 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:18.237673 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:18.237892 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:18.237895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:18.237903 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:18.238412 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.238443 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.239161 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.239162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.239945 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.239948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.240384 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.240385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.240514 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.241090 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:18.241091 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:18.241137 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:18.241171 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:18.241408 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:18.241415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:18.241425 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:38:18.241507 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:38:18.241631 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260809 00:38:18.242619 18336 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34927
I20260809 00:38:18.242735 18336 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:18.243090 18336 heartbeater.cc:511] Master 127.7.105.126:34927 requested a full tablet report, sending...
I20260809 00:38:18.243772 18125 ts_manager.cc:194] Registered new tserver with Master: e2e060a2320e4e3a8b551c2e429a8a84 (127.7.105.65:43953)
I20260809 00:38:18.244486 18125 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.65:48677
I20260809 00:38:18.245576 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18192
I20260809 00:38:18.245663 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:38:18.245898 7589 external_mini_cluster.cc:1428] Setting key 77c22089a17fb95229e06ec02e4a369c
I20260809 00:38:18.246874 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--builtin_ntp_servers=127.7.105.84:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:38:18.332492 18340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.332696 18340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.332717 18340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.333933 18340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.334070 18340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.334098 18340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:38:18.335542 18340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.335798 18340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.336181 18340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.341369 18346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.341486 18348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.341545 18340 server_base.cc:1034] running on GCE node
W20260809 00:38:18.341378 18351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.341835 18340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.342111 18340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.343288 18340 hybrid_clock.cc:648] HybridClock initialized: now 1786235898343262 us; error 36 us; skew 500 ppm
I20260809 00:38:18.344594 18340 webserver.cc:460] Webserver started at http://127.7.105.66:45647/ using document root <none> and password file <none>
I20260809 00:38:18.344839 18340 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.344910 18340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.345055 18340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.346174 18340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "1126427dcc5841578ab15b56e3da90d7"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5142c86a5675b65c302aa2cb9332c557"
server_key_iv: "21073e20d8d9a28bbfe9a0d61048f9fb"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.346545 18340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "1126427dcc5841578ab15b56e3da90d7"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5142c86a5675b65c302aa2cb9332c557"
server_key_iv: "21073e20d8d9a28bbfe9a0d61048f9fb"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.347877 18340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:38:18.348834 18358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.349068 18340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:38:18.349159 18340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "1126427dcc5841578ab15b56e3da90d7"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "5142c86a5675b65c302aa2cb9332c557"
server_key_iv: "21073e20d8d9a28bbfe9a0d61048f9fb"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.349223 18340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.382520 18340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.382863 18340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.383144 18340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.383401 18340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:18.383836 18340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:18.383898 18340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.383929 18340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:18.383989 18340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.391363 18340 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39767
I20260809 00:38:18.391420 18488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39767 every 8 connection(s)
I20260809 00:38:18.391901 18340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:38:18.393960 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18340
I20260809 00:38:18.394055 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:38:18.394259 7589 external_mini_cluster.cc:1428] Setting key 7b68e2407c5f9c761a0088e1b918ef7d
I20260809 00:38:18.395380 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--builtin_ntp_servers=127.7.105.84:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 00:38:18.398036 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.392893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:36285 (local address 127.7.105.126:34927)
0809 00:38:18.392978 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.392981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.393288 (+ 307us) server_negotiation.cc:407] Connection header received
0809 00:38:18.393798 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.393799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.393813 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.393893 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.394775 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.395395 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.396515 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.396730 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.396804 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.397754 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.397755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.397756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.397760 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.397776 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.397780 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.397828 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.397922 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.397954 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:38:18.398257 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.392879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34927 (local address 127.7.105.66:36285)
0809 00:38:18.393142 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:18.393180 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:18.393680 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:18.394050 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:18.394055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:18.394067 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:18.394664 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.394670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.395503 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.395503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.396412 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.396415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.396825 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.396826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.396959 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.397578 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:18.397578 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:18.397610 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:18.397633 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:18.397827 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:18.397833 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:18.397836 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:38:18.397925 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:38:18.398045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":125,"threads_started":1}
I20260809 00:38:18.398825 18490 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34927
I20260809 00:38:18.398944 18490 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:18.399135 18490 heartbeater.cc:511] Master 127.7.105.126:34927 requested a full tablet report, sending...
I20260809 00:38:18.399565 18125 ts_manager.cc:194] Registered new tserver with Master: 1126427dcc5841578ab15b56e3da90d7 (127.7.105.66:39767)
I20260809 00:38:18.400162 18125 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.66:36285
W20260809 00:38:18.481197 18494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.481427 18494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.481452 18494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.482574 18494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.482612 18494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.482637 18494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:38:18.483947 18494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.484221 18494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.484581 18494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:38:18.490679 18494 server_base.cc:1034] running on GCE node
W20260809 00:38:18.490830 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.491052 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.491076 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.491539 18494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.491851 18494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.493027 18494 hybrid_clock.cc:648] HybridClock initialized: now 1786235898492941 us; error 103 us; skew 500 ppm
I20260809 00:38:18.494547 18494 webserver.cc:460] Webserver started at http://127.7.105.67:35859/ using document root <none> and password file <none>
I20260809 00:38:18.494835 18494 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.494990 18494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.495160 18494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.496573 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "596447f737d7437992d29f21bd12b704"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "49947f432fd23929181cbd03171a645f"
server_key_iv: "75cc2fa95c897d6c70c31539b3e32cb4"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.496999 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "596447f737d7437992d29f21bd12b704"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "49947f432fd23929181cbd03171a645f"
server_key_iv: "75cc2fa95c897d6c70c31539b3e32cb4"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.498389 18494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:38:18.499276 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.499450 18494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:18.499557 18494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "596447f737d7437992d29f21bd12b704"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "49947f432fd23929181cbd03171a645f"
server_key_iv: "75cc2fa95c897d6c70c31539b3e32cb4"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.499648 18494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.518693 18494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.519063 18494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.519234 18494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.519500 18494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:18.519873 18494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:18.519937 18494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.519979 18494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:18.520008 18494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.526484 18494 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:37033
I20260809 00:38:18.526568 18629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:37033 every 8 connection(s)
I20260809 00:38:18.526948 18494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:38:18.531811 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18494
I20260809 00:38:18.531890 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:38:18.532197 7589 external_mini_cluster.cc:1428] Setting key 63be556905f81303323697293d304e75
I20260809 00:38:18.532312 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.527742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36761 (local address 127.7.105.126:34927)
0809 00:38:18.527813 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.527815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.528087 (+ 272us) server_negotiation.cc:407] Connection header received
0809 00:38:18.528636 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.528637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.528647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.528712 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.529369 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.529874 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.530832 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.531013 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.531078 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.531980 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.531982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.531982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.531988 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.532006 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.532010 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.532093 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.532198 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.532225 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:38:18.532575 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.527709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34927 (local address 127.7.105.67:36761)
0809 00:38:18.527931 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:18.527977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:18.528547 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:18.528766 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:18.528770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:18.528778 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:18.529290 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.529294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.529960 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.529961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.530754 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:18.530756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:18.531093 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:18.531094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:18.531206 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:18.531778 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:18.531778 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:18.531808 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:18.531832 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:18.532102 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:18.532107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:18.532110 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:38:18.532180 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:38:18.532291 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
I20260809 00:38:18.533087 18630 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34927
I20260809 00:38:18.533205 18630 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:18.533413 18630 heartbeater.cc:511] Master 127.7.105.126:34927 requested a full tablet report, sending...
I20260809 00:38:18.533815 18125 ts_manager.cc:194] Registered new tserver with Master: 596447f737d7437992d29f21bd12b704 (127.7.105.67:37033)
I20260809 00:38:18.534247 18125 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.67:36761
I20260809 00:38:18.544394 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:38:18.546561 18168 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:18.546034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.7.105.126:34927)
0809 00:38:18.546084 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.546086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.546139 (+ 53us) server_negotiation.cc:407] Connection header received
0809 00:38:18.546196 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.546197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.546205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.546256 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.546459 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46768: BlockingRecv error: recv got EOF from 127.0.0.1:46768 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:38:18.546636 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18192
I20260809 00:38:18.554692 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18340
I20260809 00:38:18.561394 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18494
I20260809 00:38:18.567828 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18088
2026-08-09T00:38:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (656 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T00:38:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:18Z Disabled control of system clock
I20260809 00:38:18.599151 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:35057
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:35057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:38:18.684247 18643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.684460 18643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.684484 18643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.685667 18643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:18.685712 18643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.685725 18643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.685738 18643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:18.685750 18643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:18.687150 18643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:35057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.687419 18643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.687736 18643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.693150 18649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.693073 18650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.693189 18652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.693322 18643 server_base.cc:1034] running on GCE node
I20260809 00:38:18.693595 18643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.693861 18643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.695019 18643 hybrid_clock.cc:648] HybridClock initialized: now 1786235898695000 us; error 34 us; skew 500 ppm
I20260809 00:38:18.696382 18643 webserver.cc:460] Webserver started at http://127.7.105.126:37077/ using document root <none> and password file <none>
I20260809 00:38:18.696645 18643 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.696691 18643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.696797 18643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.697854 18643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "3494ad1fc8304c979a7c2f3838dae313"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "0973d7b2accc35b6e2bba63d7ca86c0c"
server_key_iv: "d88703fae4c9ec647d679511c3cfc255"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.698186 18643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "3494ad1fc8304c979a7c2f3838dae313"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "0973d7b2accc35b6e2bba63d7ca86c0c"
server_key_iv: "d88703fae4c9ec647d679511c3cfc255"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.699560 18643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:38:18.700309 18658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.700507 18643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:18.700616 18643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "3494ad1fc8304c979a7c2f3838dae313"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "0973d7b2accc35b6e2bba63d7ca86c0c"
server_key_iv: "d88703fae4c9ec647d679511c3cfc255"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.700673 18643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.703950 18643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.704228 18643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.704335 18643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.708279 18643 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:35057
I20260809 00:38:18.708333 18716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:35057 every 8 connection(s)
I20260809 00:38:18.708631 18643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:38:18.709380 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:18.712544 18717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313: Bootstrap starting.
I20260809 00:38:18.713193 18717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:18.713547 18717 log.cc:824] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:18.713968 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18643
I20260809 00:38:18.714087 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:38:18.714267 7589 external_mini_cluster.cc:1428] Setting key 2359fd9886e61f9cc8918c1756824626
I20260809 00:38:18.714447 18717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313: No bootstrap required, opened a new log
I20260809 00:38:18.716343 18717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } }
I20260809 00:38:18.716501 18717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:18.716552 18717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3494ad1fc8304c979a7c2f3838dae313, State: Initialized, Role: FOLLOWER
I20260809 00:38:18.716682 18717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } }
I20260809 00:38:18.716771 18717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:38:18.716813 18717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:38:18.716854 18717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:18.717624 18717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } }
I20260809 00:38:18.717744 18717 leader_election.cc:302] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3494ad1fc8304c979a7c2f3838dae313; no voters:
I20260809 00:38:18.717820 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.714876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.7.105.126:35057)
0809 00:38:18.715150 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.715159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.715185 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:38:18.716336 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.716345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.716389 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.716522 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.717292 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.717295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.717295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.717299 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.717336 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.717347 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.717404 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.717501 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.717575 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
I20260809 00:38:18.717916 18717 leader_election.cc:288] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:38:18.718139 18717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:38:18.718291 18722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:38:18.718436 18722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [term 1 LEADER]: Becoming Leader. State: Replica: 3494ad1fc8304c979a7c2f3838dae313, State: Running, Role: LEADER
I20260809 00:38:18.718540 18722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } }
I20260809 00:38:18.718871 18724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3494ad1fc8304c979a7c2f3838dae313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } } }
I20260809 00:38:18.718892 18723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3494ad1fc8304c979a7c2f3838dae313. Latest consensus state: current_term: 1 leader_uuid: "3494ad1fc8304c979a7c2f3838dae313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3494ad1fc8304c979a7c2f3838dae313" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 35057 } } }
I20260809 00:38:18.718956 18724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:18.718992 18723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313 [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:18.719163 18732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:38:18.719651 18732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:38:18.724376 18732 catalog_manager.cc:1347] Generated new cluster ID: c1ba1401f21c4c0082f03d3e8ac4e709
I20260809 00:38:18.724433 18732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:38:18.727617 18732 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:38:18.727782 18732 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:38:18.736769 18732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3494ad1fc8304c979a7c2f3838dae313: Generated new TSK 0
I20260809 00:38:18.736981 18732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:38:18.739050 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--builtin_ntp_servers=127.7.105.84:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:38:18.833323 18742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.833632 18742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.833664 18742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.835037 18742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.835091 18742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.835124 18742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:38:18.836670 18742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.836969 18742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.837337 18742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.842940 18749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.843026 18748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.843022 18751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.843335 18742 server_base.cc:1034] running on GCE node
I20260809 00:38:18.843652 18742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.843945 18742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.845153 18742 hybrid_clock.cc:648] HybridClock initialized: now 1786235898845137 us; error 53 us; skew 500 ppm
I20260809 00:38:18.846618 18742 webserver.cc:460] Webserver started at http://127.7.105.65:42795/ using document root <none> and password file <none>
I20260809 00:38:18.846903 18742 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.846980 18742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.847237 18742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.848477 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "390f133a17d94799bca3a56d12168148"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "c93398dec72a15850aab730072287b67"
server_key_iv: "d8682ccca7f320987c4d8a57a9de7f5a"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.848869 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "390f133a17d94799bca3a56d12168148"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "c93398dec72a15850aab730072287b67"
server_key_iv: "d8682ccca7f320987c4d8a57a9de7f5a"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.850386 18742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:38:18.851217 18757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.851414 18742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:18.851513 18742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "390f133a17d94799bca3a56d12168148"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "c93398dec72a15850aab730072287b67"
server_key_iv: "d8682ccca7f320987c4d8a57a9de7f5a"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.851598 18742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.855186 18742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.855487 18742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.855635 18742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.855858 18742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:18.856244 18742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:18.856298 18742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.856338 18742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:18.856364 18742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.862295 18742 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:37971
I20260809 00:38:18.862336 18884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:37971 every 8 connection(s)
I20260809 00:38:18.862721 18742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:38:18.865204 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18742
I20260809 00:38:18.865317 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:38:18.865538 7589 external_mini_cluster.cc:1428] Setting key e319b2f4ed003faf2081592a5802514d
I20260809 00:38:18.866432 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.863594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:49549 (local address 127.7.105.126:35057)
0809 00:38:18.863679 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.863681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.863882 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:38:18.864484 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.864486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.864494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.864557 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.866088 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.866117 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.866117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.866123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.866147 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.866152 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.866235 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.866319 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.866339 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:38:18.866645 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.863545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:35057 (local address 127.7.105.65:49549)
0809 00:38:18.863757 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:18.863794 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:18.864355 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:18.864665 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:18.864668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:18.864677 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:18.865888 (+ 1211us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:18.865889 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:18.865928 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:18.865966 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:18.866238 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:18.866243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:18.866246 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:38:18.866307 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:38:18.866361 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260809 00:38:18.867033 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--builtin_ntp_servers=127.7.105.84:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 00:38:18.868738 18885 heartbeater.cc:347] Connected to a master server at 127.7.105.126:35057
I20260809 00:38:18.868862 18885 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:18.869056 18885 heartbeater.cc:511] Master 127.7.105.126:35057 requested a full tablet report, sending...
I20260809 00:38:18.869689 18679 ts_manager.cc:194] Registered new tserver with Master: 390f133a17d94799bca3a56d12168148 (127.7.105.65:37971)
W20260809 00:38:18.952757 18889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:18.953029 18889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:18.953069 18889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:18.954344 18889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:18.954399 18889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:18.954443 18889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:38:18.956092 18889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:18.956467 18889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:18.956836 18889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:18.962325 18894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:18.962285 18897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.962348 18889 server_base.cc:1034] running on GCE node
W20260809 00:38:18.962273 18895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:18.962795 18889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:18.963052 18889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:18.964167 18889 hybrid_clock.cc:648] HybridClock initialized: now 1786235898964150 us; error 30 us; skew 500 ppm
I20260809 00:38:18.965683 18889 webserver.cc:460] Webserver started at http://127.7.105.66:33747/ using document root <none> and password file <none>
I20260809 00:38:18.965970 18889 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:18.966032 18889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:18.966215 18889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:18.967396 18889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "4a9b1724d5a1466d87ab2e68c567900a"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "1d4efe710ff629917cdeced343f4f797"
server_key_iv: "1b6240f3c6454e08ce4d65560e048e77"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.967792 18889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "4a9b1724d5a1466d87ab2e68c567900a"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "1d4efe710ff629917cdeced343f4f797"
server_key_iv: "1b6240f3c6454e08ce4d65560e048e77"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.969311 18889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:38:18.970216 18903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.970458 18889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:38:18.970592 18889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "4a9b1724d5a1466d87ab2e68c567900a"
format_stamp: "Formatted at 2026-08-09 00:38:18 on dist-test-slave-1zqn"
server_key: "1d4efe710ff629917cdeced343f4f797"
server_key_iv: "1b6240f3c6454e08ce4d65560e048e77"
server_key_version: "encryptionkey@0"
I20260809 00:38:18.970672 18889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:18.974390 18889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:18.974689 18889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:18.974836 18889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:18.975104 18889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:18.975430 18889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:18.975493 18889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.975543 18889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:18.975569 18889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:18.981539 18889 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:42979
I20260809 00:38:18.981582 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:42979 every 8 connection(s)
I20260809 00:38:18.981909 18889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:38:18.983244 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 18889
I20260809 00:38:18.983325 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:38:18.983501 7589 external_mini_cluster.cc:1428] Setting key 3764d45b25dc03bb56f4e4f969deddbd
I20260809 00:38:18.984803 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--builtin_ntp_servers=127.7.105.84:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 00:38:18.985306 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.982624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:33351 (local address 127.7.105.126:35057)
0809 00:38:18.982714 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:18.982717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:18.982930 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:38:18.983467 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:18.983468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:18.983475 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:18.983534 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:18.984963 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:18.984964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:18.984965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:18.984969 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:18.984986 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:18.984991 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:18.985049 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:18.985184 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 00:38:18.985204 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:38:18.985409 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:18.982581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:35057 (local address 127.7.105.66:33351)
0809 00:38:18.982802 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:18.982838 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:18.983354 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:18.983609 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:18.983613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:18.983623 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:18.984806 (+ 1183us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:18.984807 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:18.984836 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:18.984865 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:18.985076 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:18.985082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:18.985085 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:38:18.985156 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:38:18.985219 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260809 00:38:18.985954 19029 heartbeater.cc:347] Connected to a master server at 127.7.105.126:35057
I20260809 00:38:18.986063 19029 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:18.986218 19029 heartbeater.cc:511] Master 127.7.105.126:35057 requested a full tablet report, sending...
I20260809 00:38:18.986610 18679 ts_manager.cc:194] Registered new tserver with Master: 4a9b1724d5a1466d87ab2e68c567900a (127.7.105.66:42979)
W20260809 00:38:19.068342 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:19.068619 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:19.068658 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:19.069914 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:19.069996 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:19.070039 19032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:38:19.071448 19032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:19.071734 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:19.072085 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:19.077455 19040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:19.077508 19032 server_base.cc:1034] running on GCE node
W20260809 00:38:19.077513 19037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:19.077479 19038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:19.077852 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:19.078161 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:19.079282 19032 hybrid_clock.cc:648] HybridClock initialized: now 1786235899079255 us; error 33 us; skew 500 ppm
I20260809 00:38:19.080653 19032 webserver.cc:460] Webserver started at http://127.7.105.67:39477/ using document root <none> and password file <none>
I20260809 00:38:19.080896 19032 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:19.080945 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:19.081081 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:19.082185 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "c881e2b20cc8485dbf86546f31adfb94"
format_stamp: "Formatted at 2026-08-09 00:38:19 on dist-test-slave-1zqn"
server_key: "4de5197ccc1e265dc729b63872ea0a13"
server_key_iv: "8871be8ebf236ef11665685527a833f7"
server_key_version: "encryptionkey@0"
I20260809 00:38:19.082551 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "c881e2b20cc8485dbf86546f31adfb94"
format_stamp: "Formatted at 2026-08-09 00:38:19 on dist-test-slave-1zqn"
server_key: "4de5197ccc1e265dc729b63872ea0a13"
server_key_iv: "8871be8ebf236ef11665685527a833f7"
server_key_version: "encryptionkey@0"
I20260809 00:38:19.083907 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:38:19.084781 19048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:19.085021 19032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:19.085125 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "c881e2b20cc8485dbf86546f31adfb94"
format_stamp: "Formatted at 2026-08-09 00:38:19 on dist-test-slave-1zqn"
server_key: "4de5197ccc1e265dc729b63872ea0a13"
server_key_iv: "8871be8ebf236ef11665685527a833f7"
server_key_version: "encryptionkey@0"
I20260809 00:38:19.085201 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:19.088675 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:19.088943 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:19.089075 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:19.089293 19032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:19.089653 19032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:19.089690 19032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:19.089710 19032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:19.089723 19032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:19.095623 19032 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:43853
I20260809 00:38:19.095705 19169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:43853 every 8 connection(s)
I20260809 00:38:19.096014 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:38:19.099201 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:19.096835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:49523 (local address 127.7.105.126:35057)
0809 00:38:19.096905 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:19.096908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:19.097237 (+ 329us) server_negotiation.cc:407] Connection header received
0809 00:38:19.097687 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:19.097688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:19.097693 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:19.097747 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:19.099027 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:19.099028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:19.099028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:38:19.099031 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:19.099041 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:38:19.099045 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:19.099086 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:19.099133 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 00:38:19.099145 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:38:19.099424 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:19.096823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:35057 (local address 127.7.105.67:49523)
0809 00:38:19.097105 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:19.097141 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:19.097602 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:19.097811 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:19.097815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:19.097823 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:19.098882 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:38:19.098882 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:19.098911 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:38:19.098940 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:19.099086 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:19.099089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:19.099091 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:38:19.099143 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:38:19.099190 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":109,"threads_started":1}
I20260809 00:38:19.100026 19170 heartbeater.cc:347] Connected to a master server at 127.7.105.126:35057
I20260809 00:38:19.100157 19170 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:19.100327 19170 heartbeater.cc:511] Master 127.7.105.126:35057 requested a full tablet report, sending...
I20260809 00:38:19.100730 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 19032
I20260809 00:38:19.100740 18679 ts_manager.cc:194] Registered new tserver with Master: c881e2b20cc8485dbf86546f31adfb94 (127.7.105.67:43853)
I20260809 00:38:19.100824 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:38:19.101058 7589 external_mini_cluster.cc:1428] Setting key 67cf3356e6340c77ed039c1258c02039
I20260809 00:38:19.102015 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:38:19.103845 18721 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:38:19.103346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.7.105.126:35057)
0809 00:38:19.103403 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:19.103405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:19.103560 (+ 155us) server_negotiation.cc:407] Connection header received
0809 00:38:19.103598 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:19.103598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:19.103602 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:19.103647 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:19.103783 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38854: BlockingRecv error: recv got EOF from 127.0.0.1:38854 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:38:19.103905 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18742
I20260809 00:38:19.113775 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18889
I20260809 00:38:19.122380 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 19032
I20260809 00:38:19.129359 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 18643
2026-08-09T00:38:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (561 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:38:19 dist-test-slave-1zqn krb5kdc[19181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:38:19 dist-test-slave-1zqn krb5kdc[19181](info): set up 2 sockets
Aug 09 00:38:19 dist-test-slave-1zqn krb5kdc[19181](info): commencing operation
krb5kdc: starting...
W20260809 00:38:20.263038 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:38:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:38:20.393306 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40369
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:20.481386 19194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:20.481624 19194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:20.481650 19194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:20.482738 19194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:38:20.482790 19194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:20.482807 19194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:20.482821 19194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:38:20.482837 19194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:38:20.484305 19194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:20.484521 19194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:20.484704 19194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:20.489873 19200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:20.490010 19202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:20.489998 19194 server_base.cc:1034] running on GCE node
W20260809 00:38:20.490037 19199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:20.490450 19194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:20.490728 19194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:20.491884 19194 hybrid_clock.cc:648] HybridClock initialized: now 1786235900491873 us; error 32 us; skew 500 ppm
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:20.494876 19194 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:38:20.495370 19194 webserver.cc:460] Webserver started at http://127.7.105.126:35617/ using document root <none> and password file <none>
I20260809 00:38:20.495602 19194 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:20.495647 19194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:20.495745 19194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:20.496960 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "35c95f811ee940008c2782233749c2bd"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "c05ee8177f276eef3ab4a2b913483ade"
server_key_iv: "5c2e143758d657708bd44e83edc38b7f"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.497304 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "35c95f811ee940008c2782233749c2bd"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "c05ee8177f276eef3ab4a2b913483ade"
server_key_iv: "5c2e143758d657708bd44e83edc38b7f"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.499114 19194 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:38:20.500324 19211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.500519 19194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:38:20.500640 19194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "35c95f811ee940008c2782233749c2bd"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "c05ee8177f276eef3ab4a2b913483ade"
server_key_iv: "5c2e143758d657708bd44e83edc38b7f"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.500721 19194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:20.517979 19194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:20.520133 19194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:20.520257 19194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:20.524580 19194 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:40369
I20260809 00:38:20.524636 19267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:40369 every 8 connection(s)
I20260809 00:38:20.525019 19194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:38:20.526094 19268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:20.528393 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: Bootstrap starting.
I20260809 00:38:20.529016 19268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:20.529349 19268 log.cc:824] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:20.529419 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 19194
I20260809 00:38:20.529562 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:38:20.529757 7589 external_mini_cluster.cc:1428] Setting key ea74c23d550d44c5109e8893396210f4
I20260809 00:38:20.530194 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: No bootstrap required, opened a new log
I20260809 00:38:20.531939 19268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } }
I20260809 00:38:20.532146 19268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:20.532197 19268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c95f811ee940008c2782233749c2bd, State: Initialized, Role: FOLLOWER
I20260809 00:38:20.532327 19268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } }
I20260809 00:38:20.532414 19268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:38:20.532450 19268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:38:20.532502 19268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:20.533495 19268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } }
I20260809 00:38:20.533587 19268 leader_election.cc:302] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c95f811ee940008c2782233749c2bd; no voters:
I20260809 00:38:20.533759 19268 leader_election.cc:288] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:38:20.533797 19274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:38:20.533914 19274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [term 1 LEADER]: Becoming Leader. State: Replica: 35c95f811ee940008c2782233749c2bd, State: Running, Role: LEADER
I20260809 00:38:20.533994 19274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } }
I20260809 00:38:20.534034 19268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:38:20.534319 19274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c95f811ee940008c2782233749c2bd. Latest consensus state: current_term: 1 leader_uuid: "35c95f811ee940008c2782233749c2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } } }
I20260809 00:38:20.534384 19274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:20.534919 19279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:38:20.535399 19276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35c95f811ee940008c2782233749c2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c95f811ee940008c2782233749c2bd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40369 } } }
I20260809 00:38:20.535455 19279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:38:20.535481 19276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd [sys.catalog]: This master's current role is: LEADER
I20260809 00:38:20.537082 19279 catalog_manager.cc:1347] Generated new cluster ID: c3a2fd9862fe4640baf4551c1f0dc071
I20260809 00:38:20.537137 19279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:20.542038 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:20.530299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.7.105.126:40369)
0809 00:38:20.530496 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:20.530504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:20.530529 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:38:20.531929 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:20.531937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:20.531991 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:20.532195 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:20.532984 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.534125 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.535208 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.535849 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.535987 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:20.538594 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:20.538599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:20.538599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:20.538611 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:20.540464 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.540843 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.540847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.540847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.540968 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.541212 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.541213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.541213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.541448 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:20.541624 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:20.541694 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 00:38:20.541841 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260809 00:38:20.556735 19279 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:38:20.557350 19279 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:38:20.559860 19279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: Generated new TSK 0
I20260809 00:38:20.560127 19279 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:38:20.591814 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:20.675748 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:20.675961 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:20.675987 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:20.677301 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:20.677349 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:20.677376 19298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:38:20.678864 19298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:20.679143 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:20.679392 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:20.684558 19305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:20.684561 19307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:20.684568 19304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:20.684643 19298 server_base.cc:1034] running on GCE node
I20260809 00:38:20.685063 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:20.685324 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:20.686492 19298 hybrid_clock.cc:648] HybridClock initialized: now 1786235900686480 us; error 43 us; skew 500 ppm
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:20.689008 19298 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:38:20.689435 19298 webserver.cc:460] Webserver started at http://127.7.105.65:37503/ using document root <none> and password file <none>
I20260809 00:38:20.689666 19298 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:20.689734 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:20.689888 19298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:20.690949 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "7282e54875af4ad886aeea4ecfba55d7"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "9515a7edfe41f4ee6b89a4d439d1dcc5"
server_key_iv: "a22e17a9f588ec065aba171a4729171a"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.691319 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "7282e54875af4ad886aeea4ecfba55d7"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "9515a7edfe41f4ee6b89a4d439d1dcc5"
server_key_iv: "a22e17a9f588ec065aba171a4729171a"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.692778 19298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:38:20.693503 19319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.693701 19298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:38:20.693809 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "7282e54875af4ad886aeea4ecfba55d7"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "9515a7edfe41f4ee6b89a4d439d1dcc5"
server_key_iv: "a22e17a9f588ec065aba171a4729171a"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.693883 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:20.700695 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:20.702756 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:20.702914 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:20.703140 19298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:20.703436 19298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:20.703490 19298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.703539 19298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:20.703588 19298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.709676 19298 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:42955
I20260809 00:38:20.709728 19446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:42955 every 8 connection(s)
I20260809 00:38:20.710032 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:20.717579 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 19298
I20260809 00:38:20.717686 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:38:20.717907 7589 external_mini_cluster.cc:1428] Setting key bf3f8dc7d46bdec441a38efe13fbf6ef
I20260809 00:38:20.719898 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:20.710712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:41525 (local address 127.7.105.126:40369)
0809 00:38:20.710776 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:20.710778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:20.711008 (+ 230us) server_negotiation.cc:407] Connection header received
0809 00:38:20.711580 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:20.711581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:20.711595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:20.711654 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:20.712731 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.713240 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.714210 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.714411 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.714463 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:20.716700 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:20.716707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:20.716707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:20.716717 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:20.718389 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.718897 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.718898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.718898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.718968 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.719233 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.719234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.719234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.719355 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:20.719428 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:20.719713 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 00:38:20.719805 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:38:20.720135 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:20.710683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40369 (local address 127.7.105.65:41525)
0809 00:38:20.710874 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:20.710918 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:20.711484 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:20.711760 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:20.711763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:20.712443 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:20.712641 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:20.712645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.713326 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.713327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:20.714129 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:20.714131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.714485 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.714486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:20.714589 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:20.715097 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:20.715105 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:20.716585 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:20.718483 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:20.718491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:20.718493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:20.718818 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:20.719042 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:20.719043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:20.719043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:20.719157 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:20.719433 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:20.719441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:20.719583 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 00:38:20.719691 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:38:20.719852 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260809 00:38:20.720602 19447 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40369
I20260809 00:38:20.720697 19447 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:20.720906 19447 heartbeater.cc:511] Master 127.7.105.126:40369 requested a full tablet report, sending...
I20260809 00:38:20.721516 19229 ts_manager.cc:194] Registered new tserver with Master: 7282e54875af4ad886aeea4ecfba55d7 (127.7.105.65:42955)
I20260809 00:38:20.722443 19229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:41525
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:38:20.747120 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:20.836149 19452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:20.836385 19452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:20.836434 19452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:20.837595 19452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:20.837663 19452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:20.837705 19452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:38:20.839226 19452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:20.839512 19452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:20.839761 19452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:20.845230 19458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:20.845222 19461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:20.845222 19459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:20.845479 19452 server_base.cc:1034] running on GCE node
I20260809 00:38:20.845744 19452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:20.846006 19452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:20.847193 19452 hybrid_clock.cc:648] HybridClock initialized: now 1786235900847186 us; error 57 us; skew 500 ppm
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:20.849923 19452 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:38:20.850405 19452 webserver.cc:460] Webserver started at http://127.7.105.66:36277/ using document root <none> and password file <none>
I20260809 00:38:20.850679 19452 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:20.850759 19452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:20.850939 19452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:20.852137 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "9b9a3fcd9a6f46edbd506f30a74988c8"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "771412aa41f9ad8ce75b64a05849f255"
server_key_iv: "0d57bbf5950c35ec0133d592a0c9f40d"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.852526 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "9b9a3fcd9a6f46edbd506f30a74988c8"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "771412aa41f9ad8ce75b64a05849f255"
server_key_iv: "0d57bbf5950c35ec0133d592a0c9f40d"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.853938 19452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:38:20.854780 19468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.854959 19452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:38:20.855101 19452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "9b9a3fcd9a6f46edbd506f30a74988c8"
format_stamp: "Formatted at 2026-08-09 00:38:20 on dist-test-slave-1zqn"
server_key: "771412aa41f9ad8ce75b64a05849f255"
server_key_iv: "0d57bbf5950c35ec0133d592a0c9f40d"
server_key_version: "encryptionkey@0"
I20260809 00:38:20.855185 19452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:20.880275 19452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:20.882421 19452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:20.882601 19452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:20.882875 19452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:20.883231 19452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:20.883292 19452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.883342 19452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:20.883380 19452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:20.889983 19452 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:40765
I20260809 00:38:20.890028 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:40765 every 8 connection(s)
I20260809 00:38:20.890384 19452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:38:20.892894 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 19452
I20260809 00:38:20.893013 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:38:20.893205 7589 external_mini_cluster.cc:1428] Setting key 5d3e38806bd387a6cd714e8a7263d87f
Aug 09 00:38:20 dist-test-slave-1zqn krb5kdc[19181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:20.901401 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:20.891114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41215 (local address 127.7.105.126:40369)
0809 00:38:20.891216 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:20.891219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:20.891418 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:38:20.892014 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:20.892016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:20.892038 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:20.892137 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:20.893276 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.893970 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.895090 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.895310 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.895405 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:20.897944 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:20.897946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:20.897946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:20.897955 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:20.899775 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.900274 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.900275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.900276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.900350 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:20.900608 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:20.900610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:20.900610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:20.900786 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:20.900878 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:20.901183 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 00:38:20.901281 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:38:20.901435 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:20.891092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40369 (local address 127.7.105.66:41215)
0809 00:38:20.891275 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:20.891327 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:20.891895 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:20.892218 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:20.892223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:20.892872 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:20.893172 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:20.893178 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.894071 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.894073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:20.894967 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:20.894970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:20.895416 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:20.895418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:20.895580 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:20.896188 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:20.896197 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:20.897851 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:20.899876 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:20.899883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:20.899885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:20.900193 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:20.900433 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:20.900434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:20.900434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:20.900536 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:20.900898 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:20.900905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:20.901030 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 00:38:20.901129 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 00:38:20.901262 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 00:38:20.902220 19592 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40369
I20260809 00:38:20.902338 19592 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:20.902557 19592 heartbeater.cc:511] Master 127.7.105.126:40369 requested a full tablet report, sending...
I20260809 00:38:20.902983 19229 ts_manager.cc:194] Registered new tserver with Master: 9b9a3fcd9a6f46edbd506f30a74988c8 (127.7.105.66:40765)
I20260809 00:38:20.903443 19229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:41215
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:38:20.923772 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:38:21.007632 19597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:38:21.007844 19597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:38:21.007869 19597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:38:21.009117 19597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:38:21.009162 19597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:38:21.009189 19597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:38:21.010681 19597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:40369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:38:21.010958 19597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:38:21.011211 19597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:38:21.016360 19603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:21.016422 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:38:21.016381 19602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:38:21.016722 19597 server_base.cc:1034] running on GCE node
I20260809 00:38:21.016943 19597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:38:21.017154 19597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:38:21.018326 19597 hybrid_clock.cc:648] HybridClock initialized: now 1786235901018308 us; error 51 us; skew 500 ppm
Aug 09 00:38:21 dist-test-slave-1zqn krb5kdc[19181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235901, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:38:21.020901 19597 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:38:21.021418 19597 webserver.cc:460] Webserver started at http://127.7.105.67:46463/ using document root <none> and password file <none>
I20260809 00:38:21.021672 19597 fs_manager.cc:359] Metadata directory not provided
I20260809 00:38:21.021740 19597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:38:21.021894 19597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:38:21.022946 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "80297746be4345e794520a0bd9d0b5f7"
format_stamp: "Formatted at 2026-08-09 00:38:21 on dist-test-slave-1zqn"
server_key: "795e5a92f79b05a0eabd7f34ebb3ddc9"
server_key_iv: "5c8c71ac949d53d3fd4c292e0b023dc0"
server_key_version: "encryptionkey@0"
I20260809 00:38:21.023319 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "80297746be4345e794520a0bd9d0b5f7"
format_stamp: "Formatted at 2026-08-09 00:38:21 on dist-test-slave-1zqn"
server_key: "795e5a92f79b05a0eabd7f34ebb3ddc9"
server_key_iv: "5c8c71ac949d53d3fd4c292e0b023dc0"
server_key_version: "encryptionkey@0"
I20260809 00:38:21.024747 19597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:38:21.025477 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:21.025660 19597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:21.025764 19597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "80297746be4345e794520a0bd9d0b5f7"
format_stamp: "Formatted at 2026-08-09 00:38:21 on dist-test-slave-1zqn"
server_key: "795e5a92f79b05a0eabd7f34ebb3ddc9"
server_key_iv: "5c8c71ac949d53d3fd4c292e0b023dc0"
server_key_version: "encryptionkey@0"
I20260809 00:38:21.025835 19597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:38:21.058781 19597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:38:21.061012 19597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:38:21.061178 19597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:38:21.061430 19597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:38:21.061754 19597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:38:21.061808 19597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:21.061849 19597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:38:21.061877 19597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:38:21.067759 19597 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:33553
I20260809 00:38:21.067811 19741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:33553 every 8 connection(s)
I20260809 00:38:21.068194 19597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:38:21.069372 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 19597
I20260809 00:38:21.069613 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:38:21.069831 7589 external_mini_cluster.cc:1428] Setting key 537470b8ddb12f8ac097551ec199f7e3
Aug 09 00:38:21 dist-test-slave-1zqn krb5kdc[19181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235901, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:38:21.078143 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.069025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:45985 (local address 127.7.105.126:40369)
0809 00:38:21.069085 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.069088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.069281 (+ 193us) server_negotiation.cc:407] Connection header received
0809 00:38:21.069828 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.069830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.069843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.069901 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:21.071140 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.071657 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.072757 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.072975 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.073069 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.075273 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:21.075274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:21.075275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:21.075282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:21.076881 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:21.077345 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:21.077346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:21.077346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:21.077422 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:21.077634 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:21.077635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:21.077635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:21.077734 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:21.077779 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.077996 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.078069 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:38:21.078395 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.068935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40369 (local address 127.7.105.67:45985)
0809 00:38:21.069131 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.069176 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.069724 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.070025 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.070030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.070836 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:38:21.071045 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.071050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.071762 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.071763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.072663 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.072665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.073082 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.073083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.073207 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.073684 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:38:21.073693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:38:21.075140 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:38:21.076955 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:21.076962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:21.076964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:21.077279 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:21.077481 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:38:21.077483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:38:21.077483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:38:21.077591 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:38:21.077784 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:38:21.077789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:38:21.077905 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.077986 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.078129 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
I20260809 00:38:21.078998 19742 heartbeater.cc:347] Connected to a master server at 127.7.105.126:40369
I20260809 00:38:21.079165 19742 heartbeater.cc:464] Registering TS with master...
I20260809 00:38:21.079370 19742 heartbeater.cc:511] Master 127.7.105.126:40369 requested a full tablet report, sending...
I20260809 00:38:21.079811 19229 ts_manager.cc:194] Registered new tserver with Master: 80297746be4345e794520a0bd9d0b5f7 (127.7.105.67:33553)
I20260809 00:38:21.080276 19229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:45985
I20260809 00:38:21.081558 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:38:21.088752 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.082841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.7.105.126:40369)
0809 00:38:21.082912 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.082915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.082996 (+ 81us) server_negotiation.cc:407] Connection header received
0809 00:38:21.083054 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.083055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.083066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.083116 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:38:21.083627 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.084158 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.084822 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.085003 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.085054 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.086167 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:38:21.086169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:38:21.086169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:38:21.086175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:38:21.087605 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:21.087895 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:21.087896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:21.087897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:21.087948 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:38:21.088229 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:38:21.088229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:38:21.088229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:38:21.088371 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:38:21.088448 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.088575 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.088641 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:38:21.090391 19229 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:38:21.092017 19229 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:38:21.099085 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:33553 (local address 127.0.0.1:55112)
0809 00:38:21.094975 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.094985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.095033 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.095423 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.095425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.095427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:38:21.095506 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.095509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.096679 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.096681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.097853 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.097856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.098768 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.098769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.098849 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.098863 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.098933 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.098986 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":71,"thread_start_us":69,"threads_started":1}
I20260809 00:38:21.099372 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42955 (local address 127.0.0.1:54388)
0809 00:38:21.094844 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.094892 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.094970 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.095378 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.095382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.095384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:38:21.095501 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.095504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.097373 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.097374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.098429 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.098430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.099130 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.099131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.099213 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.099218 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.099264 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.099302 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260809 00:38:21.099429 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.7.105.67:33553)
0809 00:38:21.094911 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.094925 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.095087 (+ 162us) server_negotiation.cc:407] Connection header received
0809 00:38:21.095132 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.095153 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.095171 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.095280 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:38:21.095577 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.096569 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.097951 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.098586 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.098670 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.098731 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.099293 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.099344 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260809 00:38:21.099493 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:40765 (local address 127.0.0.1:60550)
0809 00:38:21.095475 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.095487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.095541 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.095757 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.095758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.095760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:38:21.095797 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.095798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.096796 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.096797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.098231 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.098234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.099260 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.099261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.099323 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.099328 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.099376 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.099415 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":68,"threads_started":1}
I20260809 00:38:21.099748 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.7.105.66:40765)
0809 00:38:21.095048 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.095053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.095609 (+ 556us) server_negotiation.cc:407] Connection header received
0809 00:38:21.095660 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.095663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.095675 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.095760 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:38:21.095855 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.096683 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.098327 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.099093 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.099161 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.099222 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.099606 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.099654 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260809 00:38:21.100239 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.094627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.7.105.65:42955)
0809 00:38:21.095141 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.095145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.095156 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:38:21.095196 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.095198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.095221 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.095362 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:38:21.096137 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.097266 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.098513 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.099039 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.099128 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.100088 (+ 960us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.100116 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.100161 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":107,"threads_started":1}
I20260809 00:38:21.100304 19669 tablet_service.cc:1467] Processing CreateTablet for tablet 445ee62efde04cdfb970b11ad16ea990 (DEFAULT_TABLE table=test-table [id=1ce92b0e8e7046bcb624105c1b589081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:38:21.100405 19522 tablet_service.cc:1467] Processing CreateTablet for tablet 445ee62efde04cdfb970b11ad16ea990 (DEFAULT_TABLE table=test-table [id=1ce92b0e8e7046bcb624105c1b589081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:38:21.100641 19669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445ee62efde04cdfb970b11ad16ea990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:21.100768 19522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445ee62efde04cdfb970b11ad16ea990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:21.100889 19371 tablet_service.cc:1467] Processing CreateTablet for tablet 445ee62efde04cdfb970b11ad16ea990 (DEFAULT_TABLE table=test-table [id=1ce92b0e8e7046bcb624105c1b589081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:38:21.101190 19371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445ee62efde04cdfb970b11ad16ea990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:38:21.103499 19761 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Bootstrap starting.
I20260809 00:38:21.104394 19761 tablet_bootstrap.cc:654] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:21.104380 19762 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Bootstrap starting.
I20260809 00:38:21.104784 19764 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Bootstrap starting.
I20260809 00:38:21.105044 19761 log.cc:824] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:21.105365 19762 tablet_bootstrap.cc:654] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:21.105527 19764 tablet_bootstrap.cc:654] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Neither blocks nor log segments found. Creating new log.
I20260809 00:38:21.105978 19761 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: No bootstrap required, opened a new log
I20260809 00:38:21.105965 19762 log.cc:824] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:21.106048 19761 ts_tablet_manager.cc:1397] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:38:21.106154 19764 log.cc:824] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:38:21.106699 19762 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: No bootstrap required, opened a new log
I20260809 00:38:21.106774 19762 ts_tablet_manager.cc:1397] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:38:21.106910 19764 tablet_bootstrap.cc:492] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: No bootstrap required, opened a new log
I20260809 00:38:21.106993 19764 ts_tablet_manager.cc:1397] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:38:21.108165 19761 raft_consensus.cc:348] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.108342 19761 raft_consensus.cc:374] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:21.108399 19761 raft_consensus.cc:729] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80297746be4345e794520a0bd9d0b5f7, State: Initialized, Role: FOLLOWER
I20260809 00:38:21.108521 19761 consensus_queue.cc:260] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.108448 19764 raft_consensus.cc:348] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.108625 19764 raft_consensus.cc:374] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:21.108671 19764 raft_consensus.cc:729] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7282e54875af4ad886aeea4ecfba55d7, State: Initialized, Role: FOLLOWER
I20260809 00:38:21.108811 19761 ts_tablet_manager.cc:1428] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:38:21.108796 19764 consensus_queue.cc:260] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.108780 19762 raft_consensus.cc:348] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.108942 19762 raft_consensus.cc:374] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:38:21.108978 19762 raft_consensus.cc:729] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9a3fcd9a6f46edbd506f30a74988c8, State: Initialized, Role: FOLLOWER
I20260809 00:38:21.108974 19742 heartbeater.cc:503] Master 127.7.105.126:40369 was elected leader, sending a full tablet report...
I20260809 00:38:21.109088 19764 ts_tablet_manager.cc:1428] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:38:21.109109 19447 heartbeater.cc:503] Master 127.7.105.126:40369 was elected leader, sending a full tablet report...
I20260809 00:38:21.109084 19762 consensus_queue.cc:260] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.109344 19762 ts_tablet_manager.cc:1428] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:38:21.109838 19592 heartbeater.cc:503] Master 127.7.105.126:40369 was elected leader, sending a full tablet report...
W20260809 00:38:21.141021 19593 tablet.cc:2369] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:38:21.203338 19772 raft_consensus.cc:482] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:38:21.203472 19772 raft_consensus.cc:504] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.203907 19772 leader_election.cc:288] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7282e54875af4ad886aeea4ecfba55d7 (127.7.105.65:42955), 80297746be4345e794520a0bd9d0b5f7 (127.7.105.67:33553)
I20260809 00:38:21.207978 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.204207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:38889 (local address 127.7.105.65:42955)
0809 00:38:21.204257 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.204259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.204433 (+ 174us) server_negotiation.cc:407] Connection header received
0809 00:38:21.204496 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.204497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.204508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.204576 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:38:21.204704 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.205446 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207013 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.207583 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207653 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.207662 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.207863 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.207903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:38:21.207978 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.204274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:58729 (local address 127.7.105.67:33553)
0809 00:38:21.204322 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.204325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.204334 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:38:21.204365 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.204365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.204374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.204473 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:38:21.204667 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.205428 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207002 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.207572 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207653 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.207662 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.207864 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.207903 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:38:21.207979 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.204125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:33553 (local address 127.7.105.66:58729)
0809 00:38:21.204210 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.204223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.204272 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.204490 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.204491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.204493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:38:21.204579 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.204580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.205547 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.205549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.206879 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.206883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.207676 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.207773 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.207778 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.207844 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.207897 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":83}
I20260809 00:38:21.208403 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.204125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42955 (local address 127.7.105.66:38889)
0809 00:38:21.204370 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.204377 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 00:38:21.204439 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:38:21.204582 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:38:21.204583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:38:21.204584 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:38:21.204644 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.204645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.205547 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.205549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.206879 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:38:21.206883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.207676 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.207677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:38:21.207773 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.207778 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:38:21.207843 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:38:21.207975 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":24,"thread_start_us":122,"threads_started":1}
I20260809 00:38:21.208493 19691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445ee62efde04cdfb970b11ad16ea990" candidate_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80297746be4345e794520a0bd9d0b5f7" is_pre_election: true
I20260809 00:38:21.208685 19691 raft_consensus.cc:2454] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b9a3fcd9a6f46edbd506f30a74988c8 in term 0.
I20260809 00:38:21.208807 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445ee62efde04cdfb970b11ad16ea990" candidate_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7282e54875af4ad886aeea4ecfba55d7" is_pre_election: true
I20260809 00:38:21.208994 19394 raft_consensus.cc:2454] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b9a3fcd9a6f46edbd506f30a74988c8 in term 0.
I20260809 00:38:21.208981 19472 leader_election.cc:302] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80297746be4345e794520a0bd9d0b5f7, 9b9a3fcd9a6f46edbd506f30a74988c8; no voters:
I20260809 00:38:21.209148 19772 raft_consensus.cc:2781] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:38:21.209214 19772 raft_consensus.cc:482] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:38:21.209251 19772 raft_consensus.cc:3037] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:21.210158 19772 raft_consensus.cc:504] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.210306 19772 leader_election.cc:288] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [CANDIDATE]: Term 1 election: Requested vote from peers 7282e54875af4ad886aeea4ecfba55d7 (127.7.105.65:42955), 80297746be4345e794520a0bd9d0b5f7 (127.7.105.67:33553)
I20260809 00:38:21.210460 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445ee62efde04cdfb970b11ad16ea990" candidate_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7282e54875af4ad886aeea4ecfba55d7"
I20260809 00:38:21.210572 19394 raft_consensus.cc:3037] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 0 FOLLOWER]: Advancing to term 1
W20260809 00:38:21.210762 19448 tablet.cc:2369] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:38:21.211378 19394 raft_consensus.cc:2454] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b9a3fcd9a6f46edbd506f30a74988c8 in term 1.
I20260809 00:38:21.211519 19691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445ee62efde04cdfb970b11ad16ea990" candidate_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80297746be4345e794520a0bd9d0b5f7"
I20260809 00:38:21.211614 19470 leader_election.cc:302] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7282e54875af4ad886aeea4ecfba55d7, 9b9a3fcd9a6f46edbd506f30a74988c8; no voters:
I20260809 00:38:21.211632 19691 raft_consensus.cc:3037] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:38:21.211716 19772 raft_consensus.cc:2781] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:38:21.211874 19772 raft_consensus.cc:686] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 1 LEADER]: Becoming Leader. State: Replica: 9b9a3fcd9a6f46edbd506f30a74988c8, State: Running, Role: LEADER
I20260809 00:38:21.211961 19772 consensus_queue.cc:237] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } }
I20260809 00:38:21.212584 19691 raft_consensus.cc:2454] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b9a3fcd9a6f46edbd506f30a74988c8 in term 1.
I20260809 00:38:21.212913 19228 catalog_manager.cc:5495] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b9a3fcd9a6f46edbd506f30a74988c8 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b9a3fcd9a6f46edbd506f30a74988c8" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 40765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 } health_report { overall_health: UNKNOWN } } }
I20260809 00:38:21.243289 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.240855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.7.105.66:40765)
0809 00:38:21.240925 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:38:21.240927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:38:21.240944 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:38:21.240986 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:38:21.240987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:38:21.241005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:38:21.241098 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:38:21.241321 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.241800 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.242539 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:38:21.242720 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:38:21.242770 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:38:21.242915 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:38:21.243020 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:38:21.243089 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:38:21.243129 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:38:21.243146 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:38:21.245281 19394 raft_consensus.cc:1261] T 445ee62efde04cdfb970b11ad16ea990 P 7282e54875af4ad886aeea4ecfba55d7 [term 1 FOLLOWER]: Refusing update from remote peer 9b9a3fcd9a6f46edbd506f30a74988c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:38:21.245301 19691 raft_consensus.cc:1261] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 1 FOLLOWER]: Refusing update from remote peer 9b9a3fcd9a6f46edbd506f30a74988c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:38:21.245725 19774 consensus_queue.cc:1035] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80297746be4345e794520a0bd9d0b5f7" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:38:21.245873 19772 consensus_queue.cc:1035] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7282e54875af4ad886aeea4ecfba55d7" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:38:21.248687 19779 mvcc.cc:205] Tried to move back new op lower bound from 7316422251497738240 to 7316422251364974592. Current Snapshot: MvccSnapshot[applied={T|T < 7316422251497738240}]
I20260809 00:38:21.251209 19228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:38:21.251317 19228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:38:21.252219 19228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35c95f811ee940008c2782233749c2bd: Sending DeleteTablet for 3 replicas of tablet 445ee62efde04cdfb970b11ad16ea990
I20260809 00:38:21.252547 19371 tablet_service.cc:1514] Processing DeleteTablet for tablet 445ee62efde04cdfb970b11ad16ea990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:38:21 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:54388
I20260809 00:38:21.252580 19522 tablet_service.cc:1514] Processing DeleteTablet for tablet 445ee62efde04cdfb970b11ad16ea990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:38:21 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:60550
I20260809 00:38:21.252612 19669 tablet_service.cc:1514] Processing DeleteTablet for tablet 445ee62efde04cdfb970b11ad16ea990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:38:21 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:55112
I20260809 00:38:21.252879 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 19298
I20260809 00:38:21.255496 19790 tablet_replica.cc:330] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: stopping tablet replica
I20260809 00:38:21.255651 19790 raft_consensus.cc:2229] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:38:21.255826 19790 raft_consensus.cc:2258] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:38:21.256078 19789 tablet_replica.cc:330] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: stopping tablet replica
I20260809 00:38:21.256198 19789 raft_consensus.cc:2229] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:38:21.256358 19789 raft_consensus.cc:2258] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:38:21.257009 19790 ts_tablet_manager.cc:1905] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:38:21.259071 19790 ts_tablet_manager.cc:1918] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:38:21.259174 19790 log.cc:1196] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/445ee62efde04cdfb970b11ad16ea990
I20260809 00:38:21.259537 19790 ts_tablet_manager.cc:1939] T 445ee62efde04cdfb970b11ad16ea990 P 80297746be4345e794520a0bd9d0b5f7: Deleting consensus metadata
I20260809 00:38:21.260118 19215 catalog_manager.cc:4841] TS 80297746be4345e794520a0bd9d0b5f7 (127.7.105.67:33553): tablet 445ee62efde04cdfb970b11ad16ea990 (table test-table [id=1ce92b0e8e7046bcb624105c1b589081]) successfully deleted
I20260809 00:38:21.260561 19789 ts_tablet_manager.cc:1905] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:38:21.262429 19789 ts_tablet_manager.cc:1918] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:38:21.262539 19789 log.cc:1196] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/445ee62efde04cdfb970b11ad16ea990
I20260809 00:38:21.262799 19789 ts_tablet_manager.cc:1939] T 445ee62efde04cdfb970b11ad16ea990 P 9b9a3fcd9a6f46edbd506f30a74988c8: Deleting consensus metadata
I20260809 00:38:21.263237 19213 catalog_manager.cc:4841] TS 9b9a3fcd9a6f46edbd506f30a74988c8 (127.7.105.66:40765): tablet 445ee62efde04cdfb970b11ad16ea990 (table test-table [id=1ce92b0e8e7046bcb624105c1b589081]) successfully deleted
W20260809 00:38:21.268426 19213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:42955 (error 108)
I20260809 00:38:21.268646 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 19452
I20260809 00:38:21.271416 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:38:21.269820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:42955 (local address 127.0.0.1:54394)
0809 00:38:21.271322 (+ 1502us) negotiation.cc:107] Waiting for socket to connect
0809 00:38:21.271371 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:42955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1476}
W20260809 00:38:21.271944 19213 catalog_manager.cc:4584] TS 7282e54875af4ad886aeea4ecfba55d7 (127.7.105.65:42955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 445ee62efde04cdfb970b11ad16ea990: Network error: Client connection negotiation failed: client connection to 127.7.105.65:42955: connect: Connection refused (error 111)
I20260809 00:38:21.279337 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 19597
I20260809 00:38:21.285254 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 19194
2026-08-09T00:38:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2157 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:38:21 dist-test-slave-1zqn krb5kdc[19794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:38:21 dist-test-slave-1zqn krb5kdc[19794](info): set up 2 sockets
Aug 09 00:38:21 dist-test-slave-1zqn krb5kdc[19794](info): commencing operation
krb5kdc: starting...
W20260809 00:38:22.387343 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:38:22 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:38:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:38:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangeradmin_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangeradmin_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangerlookup_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangerlookup_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
I20260809 00:38:22.563093 7589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjrm2oQ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/postgres -l logfile start
2026-08-09 00:38:25.275 UTC [19828] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:38:25.275 UTC [19828] LOG: listening on IPv4 address "127.7.105.84", port 50747
2026-08-09 00:38:25.279 UTC [19828] LOG: listening on Unix socket "/tmp/.s.PGSQL.50747"
2026-08-09 00:38:25.302 UTC [19831] LOG: database system was shut down at 2026-08-09 00:38:23 UTC
2026-08-09 00:38:25.306 UTC [19828] LOG: database system is ready to accept connections
I20260809 00:38:26.378113 7589 mini_postgres.cc:96] Postgres bound to 50747
2026-08-09 00:38:26.387 UTC [19840] FATAL: database "slave" does not exist
127.7.105.84:50747 - accepting connections
I20260809 00:38:26.388025 7589 mini_ranger.cc:161] Starting Ranger...
I20260809 00:38:26.404645 7589 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:38:26.934553 7589 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:38:26.934649 7589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin
2026-08-09 00:38:27,258 [I] DB FLAVOR :POSTGRES
2026-08-09 00:38:27,258 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:38:27,259 [I] Checking connection..
2026-08-09 00:38:27,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:38:27,537 [I] Checking connection passed.
2026-08-09 00:38:27,537 [I] --------- Verifying version history table ---------
2026-08-09 00:38:27,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:38:27,789 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:38:27,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:38:28,064 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:38:28,064 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:38:28,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:38:28.340 UTC [19909] WARNING: there is no transaction in progress
2026-08-09 00:38:28,348 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:38:28,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:38:28,616 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:38:28,616 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:38:28,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:38:28,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:38:29,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1zqn.c.gcp-upstream.internal','N') ;"
2026-08-09 00:38:29,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:38:29,655 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:38:29,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:38:29,915 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:38:29,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:38:30,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:38:30,446 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:38:30,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:38:31.456 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.472 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.490 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.507 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.524 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.545 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.613 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.620 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.635 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.646 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.657 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.671 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.681 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.689 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.697 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:31.704 UTC [20026] WARNING: there is no transaction in progress
2026-08-09 00:38:32,444 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:38:32,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:38:32,709 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:38:32,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:38:32,971 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:38:32,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:38:33,216 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:38:33,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:38:33,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:38:33,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1zqn.c.gcp-upstream.internal';"
2026-08-09 00:38:34,009 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:38:34,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:38:34,259 [I] DB_PATCHES have already been applied
I20260809 00:38:34.265172 7589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjrm2oQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:38:34.265273 7589 mini_ranger.cc:193] Using host: 127.7.105.84
I20260809 00:38:34.269356 7589 mini_ranger.cc:227] Ranger admin URL: http://127.7.105.84:49901
Aug 09, 2026 12:38:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:38:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:38:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:38:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:38:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:38:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:38:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:38:35.435837 7589 mini_ranger.cc:160] Time spent starting Ranger: real 9.048s user 0.001s sys 0.012s
Aug 09, 2026 12:38:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:38:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:38:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:38:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:38:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:38:57.058782 7589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangerkms_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/rangerkms_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.105.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.105.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:38:57 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235937, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:38:57.149955 7589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:38:57.161425 7589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:38:58.100500 7589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:38:58.100605 7589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms
I20260809 00:38:58.100625 7589 mini_ranger_kms.cc:227] Using postgres at 127.7.105.84:50747
2026-08-09 00:38:58,390 [I] DB FLAVOR :POSTGRES
2026-08-09 00:38:58,391 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:38:58,391 [I] Checking connection
2026-08-09 00:38:58,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:38:58,662 [I] connection success
2026-08-09 00:38:58,662 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:38:58,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:38:58,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:38:58,929 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:38:58,929 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:38:58,929 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:38:58,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:50747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:38:59,220 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:38:59.685817 7589 mini_ranger_kms.cc:310] Created kms service
I20260809 00:38:59.856930 7589 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:38:59.902562 7589 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:38:59 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235937, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:00.298839 7589 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:39:00.298990 7589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:39:00.299016 7589 mini_ranger_kms.cc:240] Using host: 127.7.105.84
I20260809 00:39:00.304116 7589 mini_ranger_kms.cc:276] Ranger KMS PID: 20233
I20260809 00:39:00.304256 7589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.105.84:33055
Aug 09, 2026 12:39:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:39:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:39:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:39:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:39:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:39:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 00:39:02.811901 7589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.662s user 0.017s sys 0.255s
I20260809 00:39:02.812029 7589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:39:02.812134 7589 mini_ranger_kms.cc:401] 127.7.105.84:33055/kms/v1/keys
Aug 09 00:39:02 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235942, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 00:39:03 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235942, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
Aug 09 00:39:03 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235942, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
Aug 09 00:39:06 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:39:06.081326 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41805
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41805
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:33055/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:06.167462 20292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:06.167677 20292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:06.167701 20292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:06.168900 20292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:39:06.168943 20292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:06.168958 20292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:06.168970 20292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:39:06.169014 20292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:39:06.170667 20292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.105.84:33055/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41805
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:06.170914 20292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:06.171156 20292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:06.176476 20302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:06.176671 20292 server_base.cc:1034] running on GCE node
W20260809 00:39:06.176438 20300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:06.176568 20299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:06.176985 20292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:06.177237 20292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:06.178375 20292 hybrid_clock.cc:648] HybridClock initialized: now 1786235946178358 us; error 35 us; skew 500 ppm
Aug 09 00:39:06 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235946, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:06.180949 20292 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:39:06.181416 20292 webserver.cc:460] Webserver started at http://127.7.105.126:44799/ using document root <none> and password file <none>
I20260809 00:39:06.181670 20292 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:06.181727 20292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:06.181874 20292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:06 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235946, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:06.247658 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f1bc449679a74e1cb019144a58ac9afa"
format_stamp: "Formatted at 2026-08-09 00:39:06 on dist-test-slave-1zqn"
server_key: "3ce41bebfa25efc5fd779b8112ba0cf6"
server_key_iv: "5e8fee3d3aa53ac44b582b30ececdb06"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:06.248171 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f1bc449679a74e1cb019144a58ac9afa"
format_stamp: "Formatted at 2026-08-09 00:39:06 on dist-test-slave-1zqn"
server_key: "3ce41bebfa25efc5fd779b8112ba0cf6"
server_key_iv: "5e8fee3d3aa53ac44b582b30ececdb06"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:06.250288 20292 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:39:06.264354 20309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:06.264628 20292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:06.264740 20292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f1bc449679a74e1cb019144a58ac9afa"
format_stamp: "Formatted at 2026-08-09 00:39:06 on dist-test-slave-1zqn"
server_key: "3ce41bebfa25efc5fd779b8112ba0cf6"
server_key_iv: "5e8fee3d3aa53ac44b582b30ececdb06"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:06.264863 20292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:06.286129 20292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:06.288717 20292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:06.288887 20292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:06.296785 20292 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:41805
I20260809 00:39:06.296811 20363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:41805 every 8 connection(s)
I20260809 00:39:06.297417 20292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:39:06.298321 20364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:06.298689 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 20292
I20260809 00:39:06.298784 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:39:06.298964 7589 external_mini_cluster.cc:1428] Setting key 16ce31c1d00fc5efd75db1ab389026dc
I20260809 00:39:06.301713 20364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: Bootstrap starting.
I20260809 00:39:06.302508 20364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:06.302842 20364 log.cc:824] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:06.303696 20364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: No bootstrap required, opened a new log
I20260809 00:39:06.306109 20364 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } }
I20260809 00:39:06.306274 20364 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:06.306335 20364 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bc449679a74e1cb019144a58ac9afa, State: Initialized, Role: FOLLOWER
I20260809 00:39:06.306499 20364 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } }
I20260809 00:39:06.306581 20364 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:39:06.306610 20364 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:39:06.306649 20364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 0 FOLLOWER]: Advancing to term 1
Aug 09 00:39:06 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:06.307297 20364 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } }
I20260809 00:39:06.307395 20364 leader_election.cc:302] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1bc449679a74e1cb019144a58ac9afa; no voters:
I20260809 00:39:06.307575 20364 leader_election.cc:288] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:39:06.307749 20371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:39:06.307852 20364 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:39:06.308007 20371 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [term 1 LEADER]: Becoming Leader. State: Replica: f1bc449679a74e1cb019144a58ac9afa, State: Running, Role: LEADER
I20260809 00:39:06.308198 20371 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } }
I20260809 00:39:06.308620 20371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1bc449679a74e1cb019144a58ac9afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } } }
I20260809 00:39:06.308763 20371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:06.308956 20371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1bc449679a74e1cb019144a58ac9afa. Latest consensus state: current_term: 1 leader_uuid: "f1bc449679a74e1cb019144a58ac9afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bc449679a74e1cb019144a58ac9afa" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41805 } } }
I20260809 00:39:06.309067 20371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:06.310396 20364 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:39:06.314433 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:06.299659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.7.105.126:41805)
0809 00:39:06.299841 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:06.299852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:06.299885 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:39:06.300490 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:06.300499 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:06.300572 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:06.301511 (+ 939us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:06.301624 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:06.302988 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:06.304096 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:06.304757 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:06.304901 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:06.307782 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:06.307786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:06.307787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:06.307799 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:06.310073 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:06.310512 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:06.310516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:06.310516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:06.310633 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:06.310869 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:06.310870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:06.310870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:06.313160 (+ 2290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:06.313529 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:06.313667 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 00:39:06.314068 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260809 00:39:06.316514 20375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:39:06.317414 20375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:39:06.325946 20375 catalog_manager.cc:1347] Generated new cluster ID: 9515d85e3d654ef59e1d147135c03939
I20260809 00:39:06.326009 20375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:39:06.346938 20375 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:39:06.347748 20375 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:39:06.351545 20375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: Generated new TSK 0
I20260809 00:39:06.351774 20375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:39:07.086659 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:33055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:07.212364 20419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:07.212667 20419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:07.212705 20419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:07.214389 20419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:07.214452 20419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:07.214493 20419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:39:07.216820 20419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.105.84:33055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:07.217070 20419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:07.217376 20419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:07.223253 20425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:07.223405 20427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:07.223261 20424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:07.224015 20419 server_base.cc:1034] running on GCE node
I20260809 00:39:07.224279 20419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:07.224542 20419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:07.227802 20419 hybrid_clock.cc:648] HybridClock initialized: now 1786235947227748 us; error 69 us; skew 500 ppm
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:07.230826 20419 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:39:07.231385 20419 webserver.cc:460] Webserver started at http://127.7.105.65:40151/ using document root <none> and password file <none>
I20260809 00:39:07.231698 20419 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:07.231781 20419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:07.231918 20419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:07.246392 20419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "f8fb9997e1a6445b834593f7a8de6638"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "c7bb5036f6f45609a487823535e6ebe4"
server_key_iv: "8aa9164571db6c03becf78d8767b46df"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.247018 20419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "f8fb9997e1a6445b834593f7a8de6638"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "c7bb5036f6f45609a487823535e6ebe4"
server_key_iv: "8aa9164571db6c03becf78d8767b46df"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.249217 20419 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 00:39:07.264194 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.264495 20419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:07.264631 20419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "f8fb9997e1a6445b834593f7a8de6638"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "c7bb5036f6f45609a487823535e6ebe4"
server_key_iv: "8aa9164571db6c03becf78d8767b46df"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.264737 20419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:07.276150 20419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:07.278820 20419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:07.279029 20419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:07.279341 20419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:07.279803 20419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:07.279908 20419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.280016 20419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:07.280134 20419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.287770 20419 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:44537
I20260809 00:39:07.288821 20419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:39:07.288085 20570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:44537 every 8 connection(s)
I20260809 00:39:07.299449 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 20419
I20260809 00:39:07.299574 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:39:07.299759 7589 external_mini_cluster.cc:1428] Setting key ed917a1cdcde7c238eada81f1fccc1ce
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:07.308219 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.291353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:33233 (local address 127.7.105.126:41805)
0809 00:39:07.295429 (+ 4076us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.295432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.295443 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:39:07.295473 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.295473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.295488 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.295571 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:07.296444 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.297339 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.298869 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.299154 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.299227 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.304480 (+ 5253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:07.304482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:07.304482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:07.304490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:07.306440 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.306998 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.306999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.307000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.307068 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.307375 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.307376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.307376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.307538 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:07.307592 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.307983 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.308104 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4035,"thread_start_us":72,"threads_started":1}
I20260809 00:39:07.308880 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.291056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41805 (local address 127.7.105.65:33233)
0809 00:39:07.291622 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.291668 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.292349 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.295716 (+ 3367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.295722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.295980 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:07.296310 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.296318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.297459 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.297460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.298763 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.298766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.299302 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.299303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.299456 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.300606 (+ 1150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:07.300617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:07.304355 (+ 3738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:07.306553 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.306562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.306566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.306890 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.307139 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.307140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.307141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.307283 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.307634 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:07.307641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:07.307816 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.308367 (+ 551us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.308549 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":80,"threads_started":1}
I20260809 00:39:07.309373 20571 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41805
I20260809 00:39:07.309479 20571 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:07.309705 20571 heartbeater.cc:511] Master 127.7.105.126:41805 requested a full tablet report, sending...
I20260809 00:39:07.310266 20326 ts_manager.cc:194] Registered new tserver with Master: f8fb9997e1a6445b834593f7a8de6638 (127.7.105.65:44537)
I20260809 00:39:07.311184 20326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:33233
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:39:07.331652 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:33055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:07.447134 20580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:07.447371 20580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:07.447412 20580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:07.448690 20580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:07.448751 20580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:07.448793 20580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:39:07.450284 20580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.105.84:33055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:07.450560 20580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:07.450834 20580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:07.456154 20588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:07.456154 20589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:07.456449 20591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:07.456840 20580 server_base.cc:1034] running on GCE node
I20260809 00:39:07.457074 20580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:07.457382 20580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:07.458571 20580 hybrid_clock.cc:648] HybridClock initialized: now 1786235947458552 us; error 29 us; skew 500 ppm
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:07.461441 20580 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:39:07.461979 20580 webserver.cc:460] Webserver started at http://127.7.105.66:44081/ using document root <none> and password file <none>
I20260809 00:39:07.462220 20580 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:07.462291 20580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:07.462427 20580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:07.476570 20580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "a39217346d2140339d1cab4a793c35ff"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "2938bdc002e92dbce99cb7afab9a1db7"
server_key_iv: "65bdb53fcde378be9220d04dba550ecc"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.477283 20580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "a39217346d2140339d1cab4a793c35ff"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "2938bdc002e92dbce99cb7afab9a1db7"
server_key_iv: "65bdb53fcde378be9220d04dba550ecc"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.484157 20580 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:07.498288 20600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.498822 20580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:07.498945 20580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "a39217346d2140339d1cab4a793c35ff"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "2938bdc002e92dbce99cb7afab9a1db7"
server_key_iv: "65bdb53fcde378be9220d04dba550ecc"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.499073 20580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:07.511514 20580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:07.514060 20580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:07.514240 20580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:07.514483 20580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:07.514892 20580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:07.514962 20580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.515002 20580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:07.515026 20580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.523667 20580 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:34973
I20260809 00:39:07.525995 20737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:34973 every 8 connection(s)
I20260809 00:39:07.526239 20580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:39:07.529489 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 20580
I20260809 00:39:07.529618 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:39:07.529837 7589 external_mini_cluster.cc:1428] Setting key 031297ea28c30796c3b69d8581b0379d
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:07.539140 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.527632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:45315 (local address 127.7.105.126:41805)
0809 00:39:07.527716 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.527719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.528332 (+ 613us) server_negotiation.cc:407] Connection header received
0809 00:39:07.529055 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.529056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.529075 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.529107 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:07.530060 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.530830 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.532433 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.532775 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.532820 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.535527 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:07.535529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:07.535529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:07.535537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:07.537369 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.537927 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.537929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.537930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.537996 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.538306 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.538307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.538308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.538464 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:07.538512 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.538937 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.539038 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:39:07.539731 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.527313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41805 (local address 127.7.105.66:45315)
0809 00:39:07.528185 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.528238 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.528917 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.529317 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.529321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.529621 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:07.529937 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.529944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.530960 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.530962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.532313 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.532317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.532924 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.532925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.533075 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.533742 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:07.533754 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:07.535395 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:07.537488 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.537496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.537499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.537826 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.538074 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.538074 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.538075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.538223 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.538566 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:07.538573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:07.538765 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.539286 (+ 521us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.539481 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":104,"threads_started":1}
I20260809 00:39:07.540237 20738 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41805
I20260809 00:39:07.540336 20738 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:07.540549 20738 heartbeater.cc:511] Master 127.7.105.126:41805 requested a full tablet report, sending...
I20260809 00:39:07.540940 20326 ts_manager.cc:194] Registered new tserver with Master: a39217346d2140339d1cab4a793c35ff (127.7.105.66:34973)
I20260809 00:39:07.541445 20326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:45315
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:39:07.578641 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:33055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:07.712594 20743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:07.712955 20743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:07.713038 20743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:07.714741 20743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:07.714964 20743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:07.715072 20743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:39:07.717216 20743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.105.84:33055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:07.717675 20743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:07.718015 20743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:07.724545 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:07.724876 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:07.726183 20743 server_base.cc:1034] running on GCE node
W20260809 00:39:07.728212 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:07.728497 20743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:07.728765 20743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:07.729928 20743 hybrid_clock.cc:648] HybridClock initialized: now 1786235947729900 us; error 38 us; skew 500 ppm
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:07.733330 20743 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:39:07.733826 20743 webserver.cc:460] Webserver started at http://127.7.105.67:34175/ using document root <none> and password file <none>
I20260809 00:39:07.734120 20743 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:07.734174 20743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:07.734297 20743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:07.752635 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "23b1dc8ea24f4f4eb97176e547961ca3"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "9a6ef1c1003bd8f753630e12d92e2f39"
server_key_iv: "50568ce8e98b7bde07ca028d733e897b"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.754405 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "23b1dc8ea24f4f4eb97176e547961ca3"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "9a6ef1c1003bd8f753630e12d92e2f39"
server_key_iv: "50568ce8e98b7bde07ca028d733e897b"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.758143 20743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260809 00:39:07.780160 20762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.780439 20743 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260809 00:39:07.780539 20743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "23b1dc8ea24f4f4eb97176e547961ca3"
format_stamp: "Formatted at 2026-08-09 00:39:07 on dist-test-slave-1zqn"
server_key: "9a6ef1c1003bd8f753630e12d92e2f39"
server_key_iv: "50568ce8e98b7bde07ca028d733e897b"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:07.780637 20743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:07.808350 20743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:07.811367 20743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:07.811581 20743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:07.811853 20743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:07.812297 20743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:07.812362 20743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.812417 20743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:07.812443 20743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:07.821265 20743 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:42727
I20260809 00:39:07.821810 20743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:39:07.824270 20884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:42727 every 8 connection(s)
Aug 09 00:39:07 dist-test-slave-1zqn krb5kdc[19794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:07.830973 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 20743
I20260809 00:39:07.831097 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:39:07.831290 7589 external_mini_cluster.cc:1428] Setting key b044dbeb2a11f2dd79492438f3040513
I20260809 00:39:07.834415 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.822976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:33597 (local address 127.7.105.126:41805)
0809 00:39:07.823029 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.823031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.823044 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:39:07.823723 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.823724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.823739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.823815 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:07.824700 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.825595 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.827008 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.827332 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.827430 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.830220 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:07.830222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:07.830223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:07.830232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:07.832760 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.833296 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.833298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.833298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.833363 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.833671 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.833672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.833672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.833808 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:07.833875 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.834239 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.834322 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:39:07.834944 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.822687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41805 (local address 127.7.105.67:33597)
0809 00:39:07.822887 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.822941 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.823586 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.823891 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.823896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.824200 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:07.824591 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.824600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.825704 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.825706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.826904 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.826907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.827440 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.827441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.827589 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.828293 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:07.828304 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:07.830111 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:07.832846 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.832855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.832858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.833207 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.833437 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:07.833438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:07.833439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:07.833598 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:07.833878 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:07.833886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:07.834074 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.834208 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.834468 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260809 00:39:07.835491 20885 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41805
I20260809 00:39:07.835589 20885 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:07.835777 20885 heartbeater.cc:511] Master 127.7.105.126:41805 requested a full tablet report, sending...
I20260809 00:39:07.836176 20326 ts_manager.cc:194] Registered new tserver with Master: 23b1dc8ea24f4f4eb97176e547961ca3 (127.7.105.67:42727)
I20260809 00:39:07.836635 20326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:33597
I20260809 00:39:07.843734 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:39:07.854308 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.845710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.7.105.126:41805)
0809 00:39:07.845767 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.845769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.845782 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:39:07.845829 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.845830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.845843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.845904 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:07.846631 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.847508 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.848632 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.848973 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.849066 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.850700 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:07.850702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:07.850703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:07.850713 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:07.852901 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.853320 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.853321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.853322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.853400 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:07.853618 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:07.853619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:07.853620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:07.853767 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:07.853879 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.854118 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.854205 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:39:07.856442 20326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:39:07.858795 20326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:39:07.882337 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.872950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:44537 (local address 127.0.0.1:42072)
0809 00:39:07.877729 (+ 4779us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.877739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.877782 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.878088 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.878091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.878094 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:07.878172 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.878174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.879534 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.879535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.880909 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.880912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.881991 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.881992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.882081 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.882095 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.882167 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.882223 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4763,"thread_start_us":140,"threads_started":1}
I20260809 00:39:07.884255 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.873145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42072 (local address 127.7.105.65:44537)
0809 00:39:07.877368 (+ 4223us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.877374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.877842 (+ 468us) server_negotiation.cc:407] Connection header received
0809 00:39:07.877887 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.877890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.877907 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.877982 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:07.878262 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.879427 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.881020 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.881795 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.881871 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.881947 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.884122 (+ 2175us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.884171 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4175,"thread_start_us":54,"threads_started":1}
I20260809 00:39:07.885536 20488 tablet_service.cc:1467] Processing CreateTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 (DEFAULT_TABLE table=test-table [id=dc10b3d97e064cd583f45f92a59efc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:07.886077 20488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385980d07e4f44a19ffcbfeee7b2c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:07.892377 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.872943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:34973 (local address 127.0.0.1:35938)
0809 00:39:07.877460 (+ 4517us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.877530 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.877613 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.888502 (+ 10889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.888504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.888507 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:07.888575 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.888577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.889866 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.889867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.891005 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.891008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.892137 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.892138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.892211 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.892216 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.892249 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.892290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4478,"thread_start_us":145,"threads_started":1}
I20260809 00:39:07.893055 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.873061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.7.105.66:34973)
0809 00:39:07.888224 (+ 15163us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.888230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.888244 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:39:07.888290 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.888292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.888310 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.888399 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:07.888673 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.889751 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.891168 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.892002 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.892814 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.892908 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.892936 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.892973 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15114,"thread_start_us":88,"threads_started":1}
I20260809 00:39:07.893894 20657 tablet_service.cc:1467] Processing CreateTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 (DEFAULT_TABLE table=test-table [id=dc10b3d97e064cd583f45f92a59efc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:07.894272 20657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385980d07e4f44a19ffcbfeee7b2c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:07.894428 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.873260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42727 (local address 127.0.0.1:34532)
0809 00:39:07.882551 (+ 9291us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.882562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.882603 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.882910 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.882911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.882913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:07.882944 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.882945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.891500 (+ 8555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.891501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.893001 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.893005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.894179 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.894180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.894243 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.894246 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.894302 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.894342 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9280,"thread_start_us":70,"threads_started":1}
I20260809 00:39:07.894743 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.873500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.7.105.67:42727)
0809 00:39:07.881768 (+ 8268us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.881774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.882666 (+ 892us) server_negotiation.cc:407] Connection header received
0809 00:39:07.882720 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.882723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.882739 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.882811 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:07.883017 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.891375 (+ 8358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.893159 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.893991 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.894066 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.894142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.894612 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.894657 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8217,"thread_start_us":81,"threads_started":1}
I20260809 00:39:07.895375 20813 tablet_service.cc:1467] Processing CreateTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 (DEFAULT_TABLE table=test-table [id=dc10b3d97e064cd583f45f92a59efc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:07.895735 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385980d07e4f44a19ffcbfeee7b2c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:07.898422 20905 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: Bootstrap starting.
I20260809 00:39:07.899269 20905 tablet_bootstrap.cc:654] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:07.899691 20905 log.cc:824] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:07.900877 20906 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Bootstrap starting.
I20260809 00:39:07.901896 20906 tablet_bootstrap.cc:654] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:07.902206 20906 log.cc:824] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:07.902577 20909 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Bootstrap starting.
I20260809 00:39:07.903636 20909 tablet_bootstrap.cc:654] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:07.903954 20909 log.cc:824] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:07.904800 20906 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: No bootstrap required, opened a new log
I20260809 00:39:07.904881 20906 ts_tablet_manager.cc:1397] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:39:07.907044 20906 raft_consensus.cc:348] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.907226 20906 raft_consensus.cc:374] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:07.907265 20906 raft_consensus.cc:729] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39217346d2140339d1cab4a793c35ff, State: Initialized, Role: FOLLOWER
I20260809 00:39:07.907403 20906 consensus_queue.cc:260] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.907708 20906 ts_tablet_manager.cc:1428] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:39:07.907852 20738 heartbeater.cc:503] Master 127.7.105.126:41805 was elected leader, sending a full tablet report...
I20260809 00:39:07.909011 20909 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: No bootstrap required, opened a new log
I20260809 00:39:07.909094 20909 ts_tablet_manager.cc:1397] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 00:39:07.911209 20909 raft_consensus.cc:348] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.911387 20909 raft_consensus.cc:374] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:07.911442 20909 raft_consensus.cc:729] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b1dc8ea24f4f4eb97176e547961ca3, State: Initialized, Role: FOLLOWER
I20260809 00:39:07.911561 20909 consensus_queue.cc:260] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.911818 20909 ts_tablet_manager.cc:1428] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:39:07.911962 20905 tablet_bootstrap.cc:492] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: No bootstrap required, opened a new log
I20260809 00:39:07.912035 20885 heartbeater.cc:503] Master 127.7.105.126:41805 was elected leader, sending a full tablet report...
I20260809 00:39:07.912038 20905 ts_tablet_manager.cc:1397] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260809 00:39:07.914781 20914 raft_consensus.cc:482] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:39:07.914873 20914 raft_consensus.cc:504] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.915729 20914 leader_election.cc:288] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a39217346d2140339d1cab4a793c35ff (127.7.105.66:34973), f8fb9997e1a6445b834593f7a8de6638 (127.7.105.65:44537)
I20260809 00:39:07.916594 20905 raft_consensus.cc:348] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.916757 20905 raft_consensus.cc:374] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:07.916790 20905 raft_consensus.cc:729] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8fb9997e1a6445b834593f7a8de6638, State: Initialized, Role: FOLLOWER
I20260809 00:39:07.916900 20905 consensus_queue.cc:260] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.917161 20905 ts_tablet_manager.cc:1428] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260809 00:39:07.917273 20571 heartbeater.cc:503] Master 127.7.105.126:41805 was elected leader, sending a full tablet report...
I20260809 00:39:07.919608 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.915339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:34973 (local address 127.7.105.67:44803)
0809 00:39:07.915400 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.915413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.915452 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.915782 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.915783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.915785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:07.915851 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.915853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.916993 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.916994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.918201 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.918204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.919343 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.919345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.919424 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.919428 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.919480 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.919535 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:39:07.920266 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.915413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:44803 (local address 127.7.105.66:34973)
0809 00:39:07.915453 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.915455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.915464 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:39:07.915508 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.915509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.915530 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.915591 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:07.915986 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.916767 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.918687 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.919223 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.919298 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.919308 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.920156 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.920199 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:39:07.920511 20679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385980d07e4f44a19ffcbfeee7b2c7b1" candidate_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39217346d2140339d1cab4a793c35ff" is_pre_election: true
I20260809 00:39:07.920674 20679 raft_consensus.cc:2454] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b1dc8ea24f4f4eb97176e547961ca3 in term 0.
I20260809 00:39:07.920897 20763 leader_election.cc:302] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b1dc8ea24f4f4eb97176e547961ca3, a39217346d2140339d1cab4a793c35ff; no voters:
I20260809 00:39:07.924150 20915 raft_consensus.cc:2781] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:39:07.924269 20915 raft_consensus.cc:482] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:39:07.924309 20915 raft_consensus.cc:3037] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:07.924594 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.915626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:44537 (local address 127.7.105.67:44721)
0809 00:39:07.919676 (+ 4050us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:07.919687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:07.919728 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:07.921196 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:07.921198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:07.921200 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:07.921233 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.921234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.922114 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.922115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.923157 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:07.923161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.924241 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.924242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:07.924416 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.924420 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:39:07.924477 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:39:07.924513 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4039,"thread_start_us":78,"threads_started":1}
I20260809 00:39:07.924906 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.915705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:44721 (local address 127.7.105.65:44537)
0809 00:39:07.915744 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.915746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.919853 (+ 4107us) server_negotiation.cc:407] Connection header received
0809 00:39:07.919879 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.919880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.919891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.919937 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:07.921320 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.922015 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.923262 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.923896 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.923967 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.923977 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.924795 (+ 818us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.924843 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:39:07.925331 20915 raft_consensus.cc:504] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.925384 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385980d07e4f44a19ffcbfeee7b2c7b1" candidate_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8fb9997e1a6445b834593f7a8de6638" is_pre_election: true
I20260809 00:39:07.925488 20915 leader_election.cc:288] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [CANDIDATE]: Term 1 election: Requested vote from peers a39217346d2140339d1cab4a793c35ff (127.7.105.66:34973), f8fb9997e1a6445b834593f7a8de6638 (127.7.105.65:44537)
I20260809 00:39:07.925513 20490 raft_consensus.cc:2454] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b1dc8ea24f4f4eb97176e547961ca3 in term 0.
I20260809 00:39:07.925962 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385980d07e4f44a19ffcbfeee7b2c7b1" candidate_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8fb9997e1a6445b834593f7a8de6638"
I20260809 00:39:07.926031 20490 raft_consensus.cc:3037] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:07.926914 20490 raft_consensus.cc:2454] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b1dc8ea24f4f4eb97176e547961ca3 in term 1.
I20260809 00:39:07.927101 20764 leader_election.cc:302] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b1dc8ea24f4f4eb97176e547961ca3, f8fb9997e1a6445b834593f7a8de6638; no voters:
I20260809 00:39:07.927210 20679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385980d07e4f44a19ffcbfeee7b2c7b1" candidate_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39217346d2140339d1cab4a793c35ff"
I20260809 00:39:07.927291 20679 raft_consensus.cc:3037] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:07.928225 20679 raft_consensus.cc:2454] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b1dc8ea24f4f4eb97176e547961ca3 in term 1.
I20260809 00:39:07.928633 20915 raft_consensus.cc:2781] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:39:07.928730 20915 raft_consensus.cc:686] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 23b1dc8ea24f4f4eb97176e547961ca3, State: Running, Role: LEADER
I20260809 00:39:07.928858 20915 consensus_queue.cc:237] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } }
I20260809 00:39:07.930243 20325 catalog_manager.cc:5495] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b1dc8ea24f4f4eb97176e547961ca3 (127.7.105.67). New cstate: current_term: 1 leader_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b1dc8ea24f4f4eb97176e547961ca3" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42727 } health_report { overall_health: HEALTHY } } }
I20260809 00:39:07.954416 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:07.948616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.7.105.67:42727)
0809 00:39:07.948799 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:07.948801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:07.948809 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:39:07.948851 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:07.948852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:07.948875 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:07.948939 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:39:07.949440 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.950146 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.953121 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:07.953429 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:07.953509 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:07.953772 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:39:07.953913 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:39:07.954005 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:07.954279 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 00:39:07.954304 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260809 00:39:07.961671 20490 raft_consensus.cc:1261] T 385980d07e4f44a19ffcbfeee7b2c7b1 P f8fb9997e1a6445b834593f7a8de6638 [term 1 FOLLOWER]: Refusing update from remote peer 23b1dc8ea24f4f4eb97176e547961ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:39:07.962159 20915 consensus_queue.cc:1035] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8fb9997e1a6445b834593f7a8de6638" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:39:07.964871 20679 raft_consensus.cc:1261] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 1 FOLLOWER]: Refusing update from remote peer 23b1dc8ea24f4f4eb97176e547961ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:39:07.965227 20915 consensus_queue.cc:1035] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39217346d2140339d1cab4a793c35ff" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 34973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:39:07.970044 20920 mvcc.cc:205] Tried to move back new op lower bound from 7316422442847440896 to 7316422442719006720. Current Snapshot: MvccSnapshot[applied={T|T < 7316422442847440896}]
I20260809 00:39:07.971113 20923 mvcc.cc:205] Tried to move back new op lower bound from 7316422442847440896 to 7316422442719006720. Current Snapshot: MvccSnapshot[applied={T|T < 7316422442847440896}]
I20260809 00:39:07.973276 20921 mvcc.cc:205] Tried to move back new op lower bound from 7316422442847440896 to 7316422442719006720. Current Snapshot: MvccSnapshot[applied={T|T < 7316422442847440896}]
I20260809 00:39:07.976413 20325 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:39:07.976509 20325 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:39:07.977604 20325 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1bc449679a74e1cb019144a58ac9afa: Sending DeleteTablet for 3 replicas of tablet 385980d07e4f44a19ffcbfeee7b2c7b1
I20260809 00:39:07.978034 20488 tablet_service.cc:1514] Processing DeleteTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:42072
I20260809 00:39:07.978096 20657 tablet_service.cc:1514] Processing DeleteTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:35938
I20260809 00:39:07.978863 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet 385980d07e4f44a19ffcbfeee7b2c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:34532
I20260809 00:39:07.979699 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 20419
I20260809 00:39:07.985003 20931 tablet_replica.cc:330] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: stopping tablet replica
I20260809 00:39:07.985175 20932 tablet_replica.cc:330] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: stopping tablet replica
I20260809 00:39:07.985369 20932 raft_consensus.cc:2229] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:39:07.985538 20931 raft_consensus.cc:2229] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:39:07.985931 20932 raft_consensus.cc:2258] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:39:07.985776 20931 raft_consensus.cc:2258] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:39:07.987190 20931 ts_tablet_manager.cc:1905] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:39:07.987528 20932 ts_tablet_manager.cc:1905] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:39:07.992686 20931 ts_tablet_manager.cc:1918] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:39:07.992893 20932 ts_tablet_manager.cc:1918] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:39:07.993027 20931 log.cc:1196] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/385980d07e4f44a19ffcbfeee7b2c7b1
I20260809 00:39:07.993739 20931 ts_tablet_manager.cc:1939] T 385980d07e4f44a19ffcbfeee7b2c7b1 P a39217346d2140339d1cab4a793c35ff: Deleting consensus metadata
I20260809 00:39:07.993889 20932 log.cc:1196] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/385980d07e4f44a19ffcbfeee7b2c7b1
I20260809 00:39:07.994246 20932 ts_tablet_manager.cc:1939] T 385980d07e4f44a19ffcbfeee7b2c7b1 P 23b1dc8ea24f4f4eb97176e547961ca3: Deleting consensus metadata
I20260809 00:39:07.996601 20313 catalog_manager.cc:4841] TS 23b1dc8ea24f4f4eb97176e547961ca3 (127.7.105.67:42727): tablet 385980d07e4f44a19ffcbfeee7b2c7b1 (table test-table [id=dc10b3d97e064cd583f45f92a59efc59]) successfully deleted
I20260809 00:39:07.998884 20310 catalog_manager.cc:4841] TS a39217346d2140339d1cab4a793c35ff (127.7.105.66:34973): tablet 385980d07e4f44a19ffcbfeee7b2c7b1 (table test-table [id=dc10b3d97e064cd583f45f92a59efc59]) successfully deleted
W20260809 00:39:08.012387 20311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:44537 (error 108)
I20260809 00:39:08.013473 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 20580
I20260809 00:39:08.013512 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:08.013331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:44537 (local address 127.0.0.1:42074)
0809 00:39:08.013392 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:08.013470 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:39:08.014053 20311 catalog_manager.cc:4584] TS f8fb9997e1a6445b834593f7a8de6638 (127.7.105.65:44537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 385980d07e4f44a19ffcbfeee7b2c7b1: Network error: Client connection negotiation failed: client connection to 127.7.105.65:44537: connect: Connection refused (error 111)
I20260809 00:39:08.023928 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 20743
I20260809 00:39:08.037400 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 20292
I20260809 00:39:08.045863 7589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:39:08.390496 7589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:39:08.390640 7589 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:39:08.735227 7589 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:39:08.735 UTC [19828] LOG: received smart shutdown request
2026-08-09 00:39:08.739 UTC [19828] LOG: background worker "logical replication launcher" (PID 19838) exited with exit code 1
2026-08-09 00:39:08.739 UTC [19832] LOG: shutting down
2026-08-09 00:39:08.926 UTC [19828] LOG: database system is shut down
2026-08-09T00:39:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47750 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:39:09 dist-test-slave-1zqn krb5kdc[20944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:39:09 dist-test-slave-1zqn krb5kdc[20944](info): set up 2 sockets
Aug 09 00:39:09 dist-test-slave-1zqn krb5kdc[20944](info): commencing operation
krb5kdc: starting...
W20260809 00:39:10.160620 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:39:10 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:39:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:39:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangeradmin_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangeradmin_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangerlookup_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangerlookup_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
I20260809 00:39:10.346503 7589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjrm2oQ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/postgres -l logfile start
2026-08-09 00:39:12.948 UTC [20975] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:39:12.948 UTC [20975] LOG: listening on IPv4 address "127.7.105.84", port 46271
2026-08-09 00:39:12.952 UTC [20975] LOG: listening on Unix socket "/tmp/.s.PGSQL.46271"
2026-08-09 00:39:12.974 UTC [20978] LOG: database system was shut down at 2026-08-09 00:39:11 UTC
2026-08-09 00:39:12.978 UTC [20975] LOG: database system is ready to accept connections
I20260809 00:39:14.007269 7589 mini_postgres.cc:96] Postgres bound to 46271
2026-08-09 00:39:14.014 UTC [20989] FATAL: database "slave" does not exist
127.7.105.84:46271 - accepting connections
I20260809 00:39:14.015264 7589 mini_ranger.cc:161] Starting Ranger...
I20260809 00:39:14.032817 7589 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:39:14.557968 7589 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:39:14.558053 7589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin
2026-08-09 00:39:14,851 [I] DB FLAVOR :POSTGRES
2026-08-09 00:39:14,852 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:39:14,852 [I] Checking connection..
2026-08-09 00:39:14,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:39:15,114 [I] Checking connection passed.
2026-08-09 00:39:15,114 [I] --------- Verifying version history table ---------
2026-08-09 00:39:15,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:39:15,379 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:39:15,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:39:15,653 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:39:15,653 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:39:15,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:39:15.903 UTC [21059] WARNING: there is no transaction in progress
2026-08-09 00:39:15,913 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:39:15,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:39:16,150 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:39:16,150 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:39:16,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:39:16,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:39:16,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1zqn.c.gcp-upstream.internal','N') ;"
2026-08-09 00:39:16,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:39:17,219 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:39:17,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:39:17,487 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:39:17,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:39:17,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:39:18,031 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:39:18,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:39:19.060 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.078 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.096 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.112 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.130 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.149 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.211 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.217 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.232 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.243 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.253 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.267 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.277 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.285 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.292 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19.299 UTC [21174] WARNING: there is no transaction in progress
2026-08-09 00:39:19,986 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:39:19,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:39:20,257 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:39:20,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:39:20,533 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:39:20,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:39:20,804 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:39:20,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:39:21,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:39:21,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1zqn.c.gcp-upstream.internal';"
2026-08-09 00:39:21,565 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:39:21,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:39:21,822 [I] DB_PATCHES have already been applied
I20260809 00:39:21.829028 7589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjrm2oQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:39:21.829108 7589 mini_ranger.cc:193] Using host: 127.7.105.84
I20260809 00:39:21.833374 7589 mini_ranger.cc:227] Ranger admin URL: http://127.7.105.84:48451
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:39:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:39:23.030009 7589 mini_ranger.cc:160] Time spent starting Ranger: real 9.015s user 0.000s sys 0.011s
Aug 09, 2026 12:39:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:39:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:39:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:39:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:39:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:39:45.410386 7589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangerkms_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/rangerkms_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.105.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.105.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.105.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/HTTP_127.7.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:39:45 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:39:45.511512 7589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:39:45.523171 7589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:39:46.522881 7589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:39:46.522974 7589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms
I20260809 00:39:46.522998 7589 mini_ranger_kms.cc:227] Using postgres at 127.7.105.84:46271
2026-08-09 00:39:46,718 [I] DB FLAVOR :POSTGRES
2026-08-09 00:39:46,719 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:39:46,719 [I] Checking connection
2026-08-09 00:39:46,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:39:46,992 [I] connection success
2026-08-09 00:39:46,993 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:39:46,993 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:39:46,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:39:47,276 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:39:47,276 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:39:47,276 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:39:47,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.105.84:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:39:47,579 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:39:48.132153 7589 mini_ranger_kms.cc:310] Created kms service
I20260809 00:39:48.367349 7589 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:39:48.417390 7589 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:39:48 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:48.899953 7589 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:39:48.900203 7589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjrm2oQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjrm2oQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:39:48.900238 7589 mini_ranger_kms.cc:240] Using host: 127.7.105.84
I20260809 00:39:48.906206 7589 mini_ranger_kms.cc:276] Ranger KMS PID: 21372
I20260809 00:39:48.906297 7589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.105.84:58069
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:39:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 00:39:51.401971 7589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.890s user 0.007s sys 0.174s
I20260809 00:39:51.402082 7589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:39:51.402128 7589 mini_ranger_kms.cc:401] 127.7.105.84:58069/kms/v1/keys
Aug 09 00:39:51 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 00:39:51 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
Aug 09 00:39:52 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.105.84@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
Aug 09 00:39:54 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:39:54.723367 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45867
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45867
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:54.815481 21429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:54.815706 21429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:54.815728 21429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:54.816992 21429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:39:54.817041 21429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:54.817056 21429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:54.817070 21429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:39:54.817121 21429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:39:54.818782 21429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.105.84:58069/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45867
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:54.819067 21429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:54.819332 21429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:54.825186 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:54.825376 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:54.825240 21429 server_base.cc:1034] running on GCE node
W20260809 00:39:54.825169 21437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:54.825883 21429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:54.826251 21429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:54.827425 21429 hybrid_clock.cc:648] HybridClock initialized: now 1786235994827391 us; error 57 us; skew 500 ppm
Aug 09 00:39:54 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:54.833374 21429 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:39:54.834002 21429 webserver.cc:460] Webserver started at http://127.7.105.126:46559/ using document root <none> and password file <none>
I20260809 00:39:54.834267 21429 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:54.834311 21429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:54.834463 21429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:54 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:54.900182 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "c4877c5ead874e47a73ab4c8c3059721"
format_stamp: "Formatted at 2026-08-09 00:39:54 on dist-test-slave-1zqn"
server_key: "9ae3b44ffe29e02f66f03a0f1ab4a055"
server_key_iv: "6e1cfac23d2673c1cb4450490dafa4ef"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:54.900740 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "c4877c5ead874e47a73ab4c8c3059721"
format_stamp: "Formatted at 2026-08-09 00:39:54 on dist-test-slave-1zqn"
server_key: "9ae3b44ffe29e02f66f03a0f1ab4a055"
server_key_iv: "6e1cfac23d2673c1cb4450490dafa4ef"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:54.902554 21429 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:39:54.918375 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:54.918645 21429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:39:54.918773 21429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "c4877c5ead874e47a73ab4c8c3059721"
format_stamp: "Formatted at 2026-08-09 00:39:54 on dist-test-slave-1zqn"
server_key: "9ae3b44ffe29e02f66f03a0f1ab4a055"
server_key_iv: "6e1cfac23d2673c1cb4450490dafa4ef"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:54.918879 21429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:54.950875 21429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:54.953356 21429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:54.953527 21429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:54.959071 21429 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:45867
I20260809 00:39:54.959152 21498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:45867 every 8 connection(s)
I20260809 00:39:54.959537 21429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:39:54.960464 21499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:54.961388 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 21429
I20260809 00:39:54.961505 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:39:54.961697 7589 external_mini_cluster.cc:1428] Setting key b0c99e65d403ca054cda1025309e8a7f
I20260809 00:39:54.964280 21499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Bootstrap starting.
I20260809 00:39:54.965023 21499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:54.965333 21499 log.cc:824] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:54.966130 21499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: No bootstrap required, opened a new log
I20260809 00:39:54.968611 21499 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:54.968804 21499 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:54.968847 21499 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4877c5ead874e47a73ab4c8c3059721, State: Initialized, Role: FOLLOWER
I20260809 00:39:54.969053 21499 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:54.969144 21499 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:39:54.969215 21499 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:39:54.969282 21499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:54.969871 21499 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
Aug 09 00:39:54 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:54.969998 21499 leader_election.cc:302] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4877c5ead874e47a73ab4c8c3059721; no voters:
I20260809 00:39:54.970220 21499 leader_election.cc:288] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:39:54.970252 21505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:39:54.970508 21505 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 LEADER]: Becoming Leader. State: Replica: c4877c5ead874e47a73ab4c8c3059721, State: Running, Role: LEADER
I20260809 00:39:54.970638 21499 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:39:54.970654 21505 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:54.971108 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4877c5ead874e47a73ab4c8c3059721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } } }
I20260809 00:39:54.971278 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:54.971460 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4877c5ead874e47a73ab4c8c3059721. Latest consensus state: current_term: 1 leader_uuid: "c4877c5ead874e47a73ab4c8c3059721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } } }
I20260809 00:39:54.971570 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:54.971746 21514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:39:54.972112 21499 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:39:54.972321 21514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:39:54.984328 21514 catalog_manager.cc:1347] Generated new cluster ID: e1c450c1849d4d508fc1ed363924ea1b
I20260809 00:39:54.984405 21514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:39:54.985394 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:54.962290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.7.105.126:45867)
0809 00:39:54.962488 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:54.962499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:54.962584 (+ 85us) server_negotiation.cc:407] Connection header received
0809 00:39:54.963200 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:54.963210 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:54.963282 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:54.963450 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:54.964315 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:54.965710 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:54.966836 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:54.967463 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:54.967575 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:54.970512 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:54.970517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:54.970517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:54.970530 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:54.982750 (+ 12220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:54.983426 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:54.983430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:54.983431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:54.983566 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:54.983857 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:54.983858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:54.983858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:54.984272 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:54.984871 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:54.984953 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 00:39:54.985146 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260809 00:39:55.009125 21514 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:39:55.009992 21514 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:39:55.017859 21514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Generated new TSK 0
I20260809 00:39:55.018149 21514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:39:55.799539 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:55.933251 21547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:55.933604 21547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:55.933655 21547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:55.935329 21547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:55.935388 21547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:55.935438 21547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:39:55.937572 21547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:55.938056 21547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:55.938333 21547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:55.945647 21553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:55.945730 21552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:55.946216 21547 server_base.cc:1034] running on GCE node
W20260809 00:39:55.946480 21556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:55.946791 21547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:55.947273 21547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:55.948441 21547 hybrid_clock.cc:648] HybridClock initialized: now 1786235995948410 us; error 46 us; skew 500 ppm
Aug 09 00:39:55 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:55.951514 21547 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:39:55.952105 21547 webserver.cc:460] Webserver started at http://127.7.105.65:37937/ using document root <none> and password file <none>
I20260809 00:39:55.952407 21547 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:55.952488 21547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:55.952665 21547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:55 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:55.967144 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712"
format_stamp: "Formatted at 2026-08-09 00:39:55 on dist-test-slave-1zqn"
server_key: "b1ee9862deba09cac8d55eaac2d26898"
server_key_iv: "657faf6664b00db3937efd3db9608011"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:55.967602 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712"
format_stamp: "Formatted at 2026-08-09 00:39:55 on dist-test-slave-1zqn"
server_key: "b1ee9862deba09cac8d55eaac2d26898"
server_key_iv: "657faf6664b00db3937efd3db9608011"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:55.971591 21547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 00:39:55.986501 21566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:55.986814 21547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:55.986936 21547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712"
format_stamp: "Formatted at 2026-08-09 00:39:55 on dist-test-slave-1zqn"
server_key: "b1ee9862deba09cac8d55eaac2d26898"
server_key_iv: "657faf6664b00db3937efd3db9608011"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:55.987036 21547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:56.013044 21547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:56.015565 21547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:56.015756 21547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:56.016034 21547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:56.016461 21547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:56.016517 21547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.016595 21547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:56.016626 21547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.034986 21547 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:39147
I20260809 00:39:56.035807 21547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:39:56.037353 21685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:39147 every 8 connection(s)
I20260809 00:39:56.040442 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 21547
I20260809 00:39:56.040544 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:39:56.040767 7589 external_mini_cluster.cc:1428] Setting key 9bc4b248f49023e0e2ff7480e8f842b2
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:39:56.057180 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.037027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:38541 (local address 127.7.105.126:45867)
0809 00:39:56.038227 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:56.038230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:56.038240 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:39:56.038273 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:56.038274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:56.038287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:56.038349 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:56.039045 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.039569 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.043786 (+ 4217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.044191 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.044270 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.046680 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:56.046682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:56.046683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:56.046691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:56.048766 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.049171 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.049173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.049173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.049244 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.056168 (+ 6924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.056170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.056170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.056311 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:56.056420 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:56.056949 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 00:39:56.057060 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":82,"threads_started":1}
I20260809 00:39:56.057374 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.036727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.65:38541)
0809 00:39:56.037210 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:56.037265 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:56.038003 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:56.038490 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:56.038494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:56.038727 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:56.038929 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.038933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.039679 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.039681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.042133 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.042136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.044291 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.044292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.044434 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.045026 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:56.045036 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:56.046576 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:56.048841 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.048848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.048850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.049088 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.049314 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.049315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.049316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.049455 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.056414 (+ 6959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:56.056426 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:56.056687 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 00:39:56.056871 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 00:39:56.057098 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
I20260809 00:39:56.060252 21686 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:56.060374 21686 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:56.060600 21686 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:56.061299 21460 ts_manager.cc:194] Registered new tserver with Master: c5c0afc0ef794834a0bc2d7a8eeb9712 (127.7.105.65:39147)
I20260809 00:39:56.062206 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:38541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:39:56.084220 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 00:39:56.228283 21692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:56.228665 21692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:56.228749 21692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:56.230473 21692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:56.230692 21692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:56.230785 21692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:39:56.233037 21692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:56.233525 21692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:56.233879 21692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:56.240859 21700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.241245 21699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.241464 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:56.242604 21692 server_base.cc:1034] running on GCE node
I20260809 00:39:56.242820 21692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:56.243098 21692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:56.258524 21692 hybrid_clock.cc:648] HybridClock initialized: now 1786235996258481 us; error 64 us; skew 500 ppm
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:56.262145 21692 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:39:56.262745 21692 webserver.cc:460] Webserver started at http://127.7.105.66:46443/ using document root <none> and password file <none>
I20260809 00:39:56.263080 21692 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:56.263141 21692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:56.263274 21692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:56.277164 21692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "a0697a13d2e14486a82834fb65ea8a18"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "e0be6ccd7402a6dff322dd91ee68139f"
server_key_iv: "a5c7073435ca33787e76300404fd2540"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.277635 21692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "a0697a13d2e14486a82834fb65ea8a18"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "e0be6ccd7402a6dff322dd91ee68139f"
server_key_iv: "a5c7073435ca33787e76300404fd2540"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.279251 21692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:39:56.297147 21709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.297428 21692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:39:56.297528 21692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "a0697a13d2e14486a82834fb65ea8a18"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "e0be6ccd7402a6dff322dd91ee68139f"
server_key_iv: "a5c7073435ca33787e76300404fd2540"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.297636 21692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:56.312909 21692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:56.315523 21692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:56.315757 21692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:56.316047 21692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:56.316517 21692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:56.316628 21692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.316712 21692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:56.316776 21692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.330744 21692 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:41567
I20260809 00:39:56.331418 21692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:39:56.331761 21831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:41567 every 8 connection(s)
I20260809 00:39:56.334682 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 21692
I20260809 00:39:56.334793 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:39:56.334970 7589 external_mini_cluster.cc:1428] Setting key ca9446e75e288cf5d908f7bbc44239b5
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:56.346215 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.332969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:59831 (local address 127.7.105.126:45867)
0809 00:39:56.333023 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:56.333025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:56.333481 (+ 456us) server_negotiation.cc:407] Connection header received
0809 00:39:56.334538 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:56.334540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:56.334562 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:56.334642 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:56.335686 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.336500 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.338183 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.338496 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.338543 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.342404 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:56.342406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:56.342407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:56.342416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:56.344345 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.344923 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.344924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.344924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.344992 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.345341 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.345342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.345343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.345519 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:56.345573 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:56.346002 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 00:39:56.346106 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:39:56.346794 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.332941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.66:59831)
0809 00:39:56.333281 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:56.333333 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:56.334385 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:56.334732 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:56.334737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:56.335160 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:56.335568 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.335576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.336668 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.336670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.338075 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.338078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.338704 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.338705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.338919 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.340444 (+ 1525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:56.340456 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:56.342241 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:56.344446 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.344456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.344459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.344812 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.345080 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.345080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.345081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.345252 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.345630 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:56.345637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:56.345839 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 00:39:56.346342 (+ 503us) client_negotiation.cc:240] Negotiation successful
0809 00:39:56.346535 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":174,"threads_started":1}
I20260809 00:39:56.347461 21832 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:56.347584 21832 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:56.347787 21832 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:56.348242 21460 ts_manager.cc:194] Registered new tserver with Master: a0697a13d2e14486a82834fb65ea8a18 (127.7.105.66:41567)
I20260809 00:39:56.348749 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:59831
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:39:56.371718 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:56.483134 21837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:56.483410 21837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:56.483497 21837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:56.484802 21837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:56.484874 21837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:56.484920 21837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:39:56.486791 21837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:56.487054 21837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:56.487388 21837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:56.494954 21845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.495175 21843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.495435 21842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:56.496445 21837 server_base.cc:1034] running on GCE node
I20260809 00:39:56.496827 21837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:56.497190 21837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:56.498386 21837 hybrid_clock.cc:648] HybridClock initialized: now 1786235996498367 us; error 43 us; skew 500 ppm
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:56.501430 21837 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:39:56.501892 21837 webserver.cc:460] Webserver started at http://127.7.105.67:41755/ using document root <none> and password file <none>
I20260809 00:39:56.502170 21837 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:56.502245 21837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:39:56.502389 21837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:56.518431 21837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "cb52b73d10894d23822d7d86fd3af021"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "1054baffb8bd4d36a7451dd617ef3806"
server_key_iv: "44b7dc3285107b396201ac4fa868f736"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.519294 21837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "cb52b73d10894d23822d7d86fd3af021"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "1054baffb8bd4d36a7451dd617ef3806"
server_key_iv: "44b7dc3285107b396201ac4fa868f736"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.527905 21837 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260809 00:39:56.547628 21853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.547942 21837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:56.548095 21837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "cb52b73d10894d23822d7d86fd3af021"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "1054baffb8bd4d36a7451dd617ef3806"
server_key_iv: "44b7dc3285107b396201ac4fa868f736"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.548208 21837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:56.568984 21837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:56.571641 21837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:56.571833 21837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:56.572188 21837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:56.572639 21837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:56.572703 21837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.572782 21837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:56.572815 21837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.581503 21837 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:40677
I20260809 00:39:56.581924 21837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:39:56.584208 21978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:40677 every 8 connection(s)
I20260809 00:39:56.592338 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 21837
I20260809 00:39:56.592461 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:39:56.592685 7589 external_mini_cluster.cc:1428] Setting key 3a7e90d59297671c8d6f37fc3dc5122c
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:56.599437 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.582896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:34959 (local address 127.7.105.126:45867)
0809 00:39:56.582950 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:56.582953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:56.583222 (+ 269us) server_negotiation.cc:407] Connection header received
0809 00:39:56.583799 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:56.583800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:56.583815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:56.583878 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:56.584727 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.585294 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.586447 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.586769 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.586818 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.594540 (+ 7722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:56.594542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:56.594542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:56.594552 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:56.596626 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.597461 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.597463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.597463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.597536 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:56.598518 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:56.598520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:56.598520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:56.598722 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:56.598776 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:56.599225 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 00:39:56.599327 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:39:56.604569 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:56.582902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.67:34959)
0809 00:39:56.583072 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:56.583120 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:56.583683 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:56.583968 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:56.583972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:56.584295 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:56.584618 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.584625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.585404 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.585405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.586298 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:56.586325 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:56.586940 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:56.586940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:56.587416 (+ 476us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:56.592367 (+ 4951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:56.592382 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:56.594400 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:56.596964 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.596974 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.596977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.597347 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.598243 (+ 896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:56.598244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:56.598244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:56.598415 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:56.598839 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:56.598846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:56.599041 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 00:39:56.599906 (+ 865us) client_negotiation.cc:240] Negotiation successful
0809 00:39:56.604275 (+ 4369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260809 00:39:56.605393 21980 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:56.605505 21980 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:56.605746 21980 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:56.606218 21460 ts_manager.cc:194] Registered new tserver with Master: cb52b73d10894d23822d7d86fd3af021 (127.7.105.67:40677)
I20260809 00:39:56.606777 21460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:34959
I20260809 00:39:56.619277 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:39:56.619966 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 21547
I20260809 00:39:56.663695 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 21692
I20260809 00:39:56.674124 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 21837
I20260809 00:39:56.696439 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 21429
I20260809 00:39:56.705498 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45867
--webserver_interface=127.7.105.126
--webserver_port=46559
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45867
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:39:56.790078 21983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:56.790285 21983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:56.790307 21983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:56.791456 21983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:39:56.791498 21983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:56.791513 21983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:56.791525 21983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:39:56.791581 21983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:39:56.793169 21983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45867
--ranger_config_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=46559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:56.793383 21983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:56.793601 21983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:56.798614 21990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.798684 21989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:56.798650 21992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:56.799039 21983 server_base.cc:1034] running on GCE node
I20260809 00:39:56.799288 21983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:56.799527 21983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:56.800717 21983 hybrid_clock.cc:648] HybridClock initialized: now 1786235996800702 us; error 26 us; skew 500 ppm
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:56.803177 21983 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:39:56.803635 21983 webserver.cc:460] Webserver started at http://127.7.105.126:46559/ using document root <none> and password file <none>
I20260809 00:39:56.803891 21983 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:56.803965 21983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:39:56 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:56.890333 21983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:56.891149 22002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:56.891371 21983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:56.891481 21983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "c4877c5ead874e47a73ab4c8c3059721"
format_stamp: "Formatted at 2026-08-09 00:39:54 on dist-test-slave-1zqn"
server_key: "9ae3b44ffe29e02f66f03a0f1ab4a055"
server_key_iv: "6e1cfac23d2673c1cb4450490dafa4ef"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:56.891868 21983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:56.908222 21983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:56.910115 21983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:56.910293 21983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:56.914911 21983 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:45867
I20260809 00:39:56.914959 22060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:45867 every 8 connection(s)
I20260809 00:39:56.915334 21983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:39:56.916664 22061 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:39:56.917330 22061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Bootstrap starting.
I20260809 00:39:56.923193 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 21983
I20260809 00:39:56.923491 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:39147
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:45867
--webserver_port=37937
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:39:56.925961 22061 log.cc:824] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:56.927831 22061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:39:56.928207 22061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Bootstrap complete.
I20260809 00:39:56.930734 22061 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:56.930994 22061 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4877c5ead874e47a73ab4c8c3059721, State: Initialized, Role: FOLLOWER
I20260809 00:39:56.931198 22061 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:56.931283 22061 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:39:56.931340 22061 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:39:56.931399 22061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:39:56.932521 22061 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:56.932642 22061 leader_election.cc:302] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4877c5ead874e47a73ab4c8c3059721; no voters:
I20260809 00:39:56.932837 22061 leader_election.cc:288] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:39:56.932907 22065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:39:56.933064 22065 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [term 2 LEADER]: Becoming Leader. State: Replica: c4877c5ead874e47a73ab4c8c3059721, State: Running, Role: LEADER
I20260809 00:39:56.933111 22061 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:39:56.933194 22065 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } }
I20260809 00:39:56.933532 22066 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c4877c5ead874e47a73ab4c8c3059721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } } }
I20260809 00:39:56.933656 22066 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:56.933531 22065 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4877c5ead874e47a73ab4c8c3059721. Latest consensus state: current_term: 2 leader_uuid: "c4877c5ead874e47a73ab4c8c3059721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4877c5ead874e47a73ab4c8c3059721" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45867 } } }
I20260809 00:39:56.933894 22065 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721 [sys.catalog]: This master's current role is: LEADER
I20260809 00:39:56.934109 22073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:39:56.934739 22073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:39:56.934952 22073 catalog_manager.cc:1259] Loaded cluster ID: e1c450c1849d4d508fc1ed363924ea1b
I20260809 00:39:56.935045 22073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:39:56.935094 22061 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:39:56.936021 22073 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:39:56.936256 22073 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Loaded TSK: 0
I20260809 00:39:56.938591 22073 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:39:57.041422 22063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:57.041713 22063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:57.041774 22063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:57.043470 22063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:57.043540 22063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:57.043584 22063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:39:57.046051 22063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:39147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=37937
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:57.046330 22063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:57.046613 22063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:57.053121 22100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:57.053118 22102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:57.053248 22099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:57.053606 22063 server_base.cc:1034] running on GCE node
I20260809 00:39:57.053838 22063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:57.054131 22063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:57.055325 22063 hybrid_clock.cc:648] HybridClock initialized: now 1786235997055293 us; error 44 us; skew 500 ppm
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:57.058539 22063 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:39:57.059110 22063 webserver.cc:460] Webserver started at http://127.7.105.65:37937/ using document root <none> and password file <none>
I20260809 00:39:57.059396 22063 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:57.059484 22063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:57.081941 22063 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 00:39:57.086730 22111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.086918 22063 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260809 00:39:57.087028 22063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712"
format_stamp: "Formatted at 2026-08-09 00:39:55 on dist-test-slave-1zqn"
server_key: "b1ee9862deba09cac8d55eaac2d26898"
server_key_iv: "657faf6664b00db3937efd3db9608011"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:57.087525 22063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:57.107450 22063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:57.110067 22063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:57.110216 22063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:57.110457 22063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:57.110795 22063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:57.110852 22063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.110888 22063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:57.110905 22063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.117115 22063 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:39147
I20260809 00:39:57.117570 22063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:39:57.120749 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22063
I20260809 00:39:57.121071 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:41567
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:45867
--webserver_port=46443
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:39:57.140450 22232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:39147 every 8 connection(s)
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:57.165182 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.141289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.65:53801)
0809 00:39:57.148197 (+ 6908us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:57.148256 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:57.148781 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:57.149479 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:57.149484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:57.149742 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:57.150035 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.150042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.151772 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.151773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.153052 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.153055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.154002 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.154004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.154130 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.154789 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:57.154800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:57.160424 (+ 5624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:57.163104 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.163113 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.163116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.163435 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.163768 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.163770 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.163770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.163914 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.164491 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:57.164498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:57.164657 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 00:39:57.164759 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 00:39:57.164929 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6844,"thread_start_us":113,"threads_started":1}
I20260809 00:39:57.165877 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.141388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53801 (local address 127.7.105.126:45867)
0809 00:39:57.148979 (+ 7591us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:57.148987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:57.149018 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:39:57.149192 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:57.149214 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:57.149279 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:57.149428 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:57.150184 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.151631 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.153157 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.153771 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.153873 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.160679 (+ 6806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:57.160685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:57.160685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:57.160700 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:57.162992 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.163558 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.163562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.163563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.163686 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.163998 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.163999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.163999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.164345 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:57.164424 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:57.165635 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0809 00:39:57.165740 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7527,"thread_start_us":120,"threads_started":1}
I20260809 00:39:57.166656 22233 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:57.166755 22233 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.166960 22233 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.167817 22233 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:45867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:39:57.167898 22233 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.167975 22233 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:57.168373 22233 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.168467 22233 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.168772 22233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:39:57.240172 22236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:57.240459 22236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:57.240497 22236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:57.242192 22236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:57.242245 22236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:57.242290 22236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:39:57.244410 22236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:41567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=46443
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:57.244705 22236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:57.244999 22236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:57.251590 22247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:57.251648 22244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:57.251612 22245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:57.252790 22236 server_base.cc:1034] running on GCE node
I20260809 00:39:57.253049 22236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:57.253351 22236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:57.254518 22236 hybrid_clock.cc:648] HybridClock initialized: now 1786235997254508 us; error 36 us; skew 500 ppm
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:57.257265 22236 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:39:57.257762 22236 webserver.cc:460] Webserver started at http://127.7.105.66:46443/ using document root <none> and password file <none>
I20260809 00:39:57.258026 22236 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:57.258124 22236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:57.276535 22236 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:39:57.282855 22256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.283113 22236 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260809 00:39:57.283231 22236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "a0697a13d2e14486a82834fb65ea8a18"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "e0be6ccd7402a6dff322dd91ee68139f"
server_key_iv: "a5c7073435ca33787e76300404fd2540"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:57.283651 22236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:57.301441 22236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:57.303789 22236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:57.303968 22236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:57.304286 22236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:57.304684 22236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:57.304734 22236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.304776 22236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:57.304804 22236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.312448 22236 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:41567
I20260809 00:39:57.312567 22381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:41567 every 8 connection(s)
I20260809 00:39:57.312996 22236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:57.322880 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.313862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:40413 (local address 127.7.105.126:45867)
0809 00:39:57.313945 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:57.313948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:57.314219 (+ 271us) server_negotiation.cc:407] Connection header received
0809 00:39:57.314684 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:57.314685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:57.314702 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:57.314787 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:57.315387 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.316266 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.317377 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.317691 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.317770 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.319920 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:57.319922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:57.319922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:57.319929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:57.321434 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.321863 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.321864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.321865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.321927 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.322188 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.322189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.322189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.322308 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:57.322380 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:57.322670 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 00:39:57.322783 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:39:57.322926 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22236
I20260809 00:39:57.323237 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.105.84:58069/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:40677
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:45867
--webserver_port=41755
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:39:57.323546 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.313869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.66:40413)
0809 00:39:57.314067 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:57.314112 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:57.314577 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:57.314863 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:57.314867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:57.315044 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:57.315288 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.315293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.316375 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.316376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.317274 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.317276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.317801 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.317802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.317895 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.318525 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:57.318534 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:57.319822 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:57.321537 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.321545 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.321547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.321772 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.322000 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.322001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.322001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.322118 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.322382 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:57.322390 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:57.322529 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 00:39:57.322635 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 00:39:57.322789 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260809 00:39:57.325260 22382 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:57.325376 22382 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.325604 22382 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.326334 22382 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:45867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:39:57.326421 22382 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.326503 22382 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:57.326830 22382 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.326921 22382 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.327208 22382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:39:57.450402 22385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:39:57.450719 22385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:39:57.450764 22385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:39:57.452602 22385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:39:57.452665 22385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:39:57.452710 22385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:39:57.454989 22385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.105.84:58069/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:40677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=41755
--tserver_master_addrs=127.7.105.126:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:39:57.455245 22385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:39:57.455528 22385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:39:57.461628 22392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:39:57.461634 22391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:57.462507 22385 server_base.cc:1034] running on GCE node
W20260809 00:39:57.462760 22394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:39:57.463004 22385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:39:57.463303 22385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:39:57.464457 22385 hybrid_clock.cc:648] HybridClock initialized: now 1786235997464445 us; error 28 us; skew 500 ppm
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:57.467298 22385 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:39:57.467851 22385 webserver.cc:460] Webserver started at http://127.7.105.67:41755/ using document root <none> and password file <none>
I20260809 00:39:57.468166 22385 fs_manager.cc:359] Metadata directory not provided
I20260809 00:39:57.468263 22385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:57.484689 22385 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:39:57.487027 22403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.487237 22385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:39:57.487365 22385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "cb52b73d10894d23822d7d86fd3af021"
format_stamp: "Formatted at 2026-08-09 00:39:56 on dist-test-slave-1zqn"
server_key: "1054baffb8bd4d36a7451dd617ef3806"
server_key_iv: "44b7dc3285107b396201ac4fa868f736"
server_key_version: "kuduclusterkey@0"
I20260809 00:39:57.487890 22385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:39:57.508775 22385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:39:57.511286 22385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:39:57.511461 22385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:39:57.511746 22385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:39:57.512279 22385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:39:57.512332 22385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.512378 22385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:39:57.512406 22385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:39:57.519541 22385 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:40677
I20260809 00:39:57.520036 22385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:39:57.520937 22527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:40677 every 8 connection(s)
I20260809 00:39:57.524302 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22385
Aug 09 00:39:57 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:39:57.535818 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.521925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:42439 (local address 127.7.105.126:45867)
0809 00:39:57.521984 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:57.521987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:57.522136 (+ 149us) server_negotiation.cc:407] Connection header received
0809 00:39:57.522612 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:57.522613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:57.522628 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:57.522712 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:57.523399 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.524202 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.525777 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.526086 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.526126 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.532099 (+ 5973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:57.532101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:57.532101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:57.532109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:57.534071 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.534673 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.534674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.534675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.534736 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.535020 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.535021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.535021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.535175 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:57.535223 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:57.535633 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 00:39:57.535719 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:39:57.536427 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.521578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45867 (local address 127.7.105.67:42439)
0809 00:39:57.521919 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:57.522023 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:57.522509 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:57.522765 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:57.522768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:57.522994 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:39:57.523285 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.523292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.524313 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.524314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.525666 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:57.525670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.526224 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.526224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:57.526324 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.526898 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:39:57.526909 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:39:57.531968 (+ 5059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:39:57.534205 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.534213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.534216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.534580 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.534805 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:39:57.534806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:39:57.534806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:39:57.534941 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:39:57.535278 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:39:57.535285 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:39:57.535457 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:39:57.535971 (+ 514us) client_negotiation.cc:240] Negotiation successful
0809 00:39:57.536179 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":99,"threads_started":1}
I20260809 00:39:57.536907 22528 heartbeater.cc:347] Connected to a master server at 127.7.105.126:45867
I20260809 00:39:57.537017 22528 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.537236 22528 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.537988 22528 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:45867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:39:57.538089 22528 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.538165 22528 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:57.538731 22528 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:57.538820 22528 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
W20260809 00:39:57.539121 22528 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:39:57.542309 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:57.524750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.7.105.126:45867)
0809 00:39:57.525076 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:57.525078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:57.525084 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:39:57.525104 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:57.525104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:57.525114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:57.525180 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:57.529012 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.529761 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.530857 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:57.531165 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:57.531235 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:57.539477 (+ 8242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:57.539478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:57.539478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:57.539485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:57.541115 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.541520 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.541520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.541521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.541576 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:57.541779 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:57.541780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:57.541780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:57.541873 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:57.542048 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:57.542120 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:39:57.542210 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":56,"threads_started":1}
Aug 09 00:39:58 dist-test-slave-1zqn krb5kdc[20944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:39:58.168987 22233 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:58.169171 22233 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:58.169620 22022 ts_manager.cc:194] Registered new tserver with Master: c5c0afc0ef794834a0bc2d7a8eeb9712 (127.7.105.65:39147)
I20260809 00:39:58.170718 22022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:53801
I20260809 00:39:58.327427 22382 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:58.327595 22382 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:58.327963 22022 ts_manager.cc:194] Registered new tserver with Master: a0697a13d2e14486a82834fb65ea8a18 (127.7.105.66:41567)
I20260809 00:39:58.328524 22022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:40413
Aug 09 00:39:58 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:58.539355 22528 heartbeater.cc:464] Registering TS with master...
I20260809 00:39:58.539548 22528 heartbeater.cc:511] Master 127.7.105.126:45867 requested a full tablet report, sending...
I20260809 00:39:58.539935 22022 ts_manager.cc:194] Registered new tserver with Master: cb52b73d10894d23822d7d86fd3af021 (127.7.105.67:40677)
I20260809 00:39:58.540617 22022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:42439
I20260809 00:39:58.543696 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:39:58.553541 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.544886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37826 (local address 127.7.105.126:45867)
0809 00:39:58.545064 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.545067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.545172 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:39:58.545224 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.545225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.545240 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.545343 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:39:58.546214 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.547103 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.548200 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.548538 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.548610 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.550245 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:39:58.550247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:39:58.550247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:39:58.550255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:39:58.552243 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:58.552649 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:58.552650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:58.552650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:58.552716 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:39:58.552935 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:39:58.552936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:39:58.552936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:39:58.553084 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:39:58.553185 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.553338 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.553428 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260809 00:39:58.556097 22022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:39:58.558405 22022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:39:58.567160 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.561899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:39147 (local address 127.0.0.1:38738)
0809 00:39:58.562161 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.562202 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:58.562268 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:58.562587 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:58.562590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:58.562592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:58.562668 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.562670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564148 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.564149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.565429 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.565432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.566670 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.566671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.566754 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.566770 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 00:39:58.566832 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:39:58.566884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260809 00:39:58.567500 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.562204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:41567 (local address 127.0.0.1:48374)
0809 00:39:58.563133 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.563143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:58.563190 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:58.563546 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:58.563547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:58.563549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:58.563590 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.563591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564838 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.564839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.565966 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.565969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.567182 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.567184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.567288 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.567293 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:39:58.567342 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:39:58.567388 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260809 00:39:58.567672 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.561990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.7.105.65:39147)
0809 00:39:58.562415 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.562420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.562430 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:39:58.562465 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.562467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.562484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.562573 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:58.562771 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564018 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.565533 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.566477 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.567389 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.567458 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.567486 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.567537 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":98,"threads_started":1}
I20260809 00:39:58.567857 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.562336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.7.105.66:41567)
0809 00:39:58.562494 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.562499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.563252 (+ 753us) server_negotiation.cc:407] Connection header received
0809 00:39:58.563292 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.563295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.563310 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.563377 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:58.563671 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564743 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.566062 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.567015 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.567087 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.567146 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.567699 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.567749 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260809 00:39:58.568228 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.561918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:40677 (local address 127.0.0.1:44098)
0809 00:39:58.562919 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.562930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:58.562991 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:58.563174 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:58.563175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:58.563177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:58.563222 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.563223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564767 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.564768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.565862 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.565865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.567538 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.567539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.567935 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.567940 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:39:58.568102 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 00:39:58.568149 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":177,"threads_started":1}
I20260809 00:39:58.568755 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.562481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.7.105.67:40677)
0809 00:39:58.562751 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.562756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.563011 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:39:58.563053 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.563055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.563071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.563288 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:58.563381 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.564393 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.565972 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.567183 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.568485 (+ 1302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.568562 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.568590 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.568635 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":71,"threads_started":1}
I20260809 00:39:58.568701 22309 tablet_service.cc:1467] Processing CreateTablet for tablet d0adde499c61443ab91464c68277c7ae (DEFAULT_TABLE table=test-table [id=63b97d163a39441b813790486480d72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:58.568637 22155 tablet_service.cc:1467] Processing CreateTablet for tablet d0adde499c61443ab91464c68277c7ae (DEFAULT_TABLE table=test-table [id=63b97d163a39441b813790486480d72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:58.569036 22155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0adde499c61443ab91464c68277c7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:58.569085 22309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0adde499c61443ab91464c68277c7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:58.569813 22456 tablet_service.cc:1467] Processing CreateTablet for tablet d0adde499c61443ab91464c68277c7ae (DEFAULT_TABLE table=test-table [id=63b97d163a39441b813790486480d72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:39:58.570204 22456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0adde499c61443ab91464c68277c7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:39:58.573640 22568 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Bootstrap starting.
I20260809 00:39:58.574900 22568 tablet_bootstrap.cc:654] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:58.575881 22568 log.cc:824] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:58.576475 22567 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Bootstrap starting.
I20260809 00:39:58.577641 22567 tablet_bootstrap.cc:654] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:58.578332 22567 log.cc:824] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:58.579038 22568 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: No bootstrap required, opened a new log
I20260809 00:39:58.579129 22568 ts_tablet_manager.cc:1397] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 00:39:58.582808 22568 raft_consensus.cc:348] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.582965 22568 raft_consensus.cc:374] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:58.582988 22568 raft_consensus.cc:729] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0697a13d2e14486a82834fb65ea8a18, State: Initialized, Role: FOLLOWER
I20260809 00:39:58.583333 22569 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Bootstrap starting.
I20260809 00:39:58.583566 22567 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: No bootstrap required, opened a new log
I20260809 00:39:58.583647 22567 ts_tablet_manager.cc:1397] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 00:39:58.585803 22569 tablet_bootstrap.cc:654] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Neither blocks nor log segments found. Creating new log.
I20260809 00:39:58.586983 22569 log.cc:824] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Log is configured to *not* fsync() on all Append() calls
I20260809 00:39:58.587590 22568 consensus_queue.cc:260] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.588130 22568 ts_tablet_manager.cc:1428] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20260809 00:39:58.588276 22382 heartbeater.cc:503] Master 127.7.105.126:45867 was elected leader, sending a full tablet report...
I20260809 00:39:58.588873 22567 raft_consensus.cc:348] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.589118 22567 raft_consensus.cc:374] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:58.589176 22567 raft_consensus.cc:729] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c0afc0ef794834a0bc2d7a8eeb9712, State: Initialized, Role: FOLLOWER
I20260809 00:39:58.589339 22567 consensus_queue.cc:260] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.589854 22567 ts_tablet_manager.cc:1428] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 00:39:58.589990 22233 heartbeater.cc:503] Master 127.7.105.126:45867 was elected leader, sending a full tablet report...
I20260809 00:39:58.590255 22569 tablet_bootstrap.cc:492] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: No bootstrap required, opened a new log
I20260809 00:39:58.590341 22569 ts_tablet_manager.cc:1397] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260809 00:39:58.594708 22569 raft_consensus.cc:348] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.594868 22569 raft_consensus.cc:374] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:39:58.594920 22569 raft_consensus.cc:729] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb52b73d10894d23822d7d86fd3af021, State: Initialized, Role: FOLLOWER
I20260809 00:39:58.595057 22569 consensus_queue.cc:260] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.595430 22569 ts_tablet_manager.cc:1428] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260809 00:39:58.595463 22528 heartbeater.cc:503] Master 127.7.105.126:45867 was elected leader, sending a full tablet report...
W20260809 00:39:58.619211 22235 tablet.cc:2369] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:39:58.772986 22529 tablet.cc:2369] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 00:39:58 dist-test-slave-1zqn krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for HTTP/127.7.105.84@KRBTEST.COM
I20260809 00:39:58.808353 22574 raft_consensus.cc:482] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:39:58.808585 22574 raft_consensus.cc:504] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.809227 22574 leader_election.cc:288] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5c0afc0ef794834a0bc2d7a8eeb9712 (127.7.105.65:39147), cb52b73d10894d23822d7d86fd3af021 (127.7.105.67:40677)
I20260809 00:39:58.815791 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.809692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:35825 (local address 127.7.105.65:39147)
0809 00:39:58.809773 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.809775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.810510 (+ 735us) server_negotiation.cc:407] Connection header received
0809 00:39:58.810861 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.810862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.810875 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.810909 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:58.811678 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.812520 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.814557 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.815249 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.815307 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.815318 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.815653 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.815706 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:39:58.816288 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.809387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:39147 (local address 127.7.105.66:35825)
0809 00:39:58.810436 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.810451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:58.810546 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:58.811297 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:58.811298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:58.811301 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:58.811566 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.811570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.812636 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.812637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.814433 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.814437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.815424 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.815426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.815546 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.815551 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:39:58.815956 (+ 405us) client_negotiation.cc:240] Negotiation successful
0809 00:39:58.816026 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":401,"threads_started":1}
I20260809 00:39:58.816952 22164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0adde499c61443ab91464c68277c7ae" candidate_uuid: "a0697a13d2e14486a82834fb65ea8a18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" is_pre_election: true
I20260809 00:39:58.817137 22164 raft_consensus.cc:2454] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0697a13d2e14486a82834fb65ea8a18 in term 0.
I20260809 00:39:58.818169 22262 leader_election.cc:302] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0697a13d2e14486a82834fb65ea8a18, c5c0afc0ef794834a0bc2d7a8eeb9712; no voters:
W20260809 00:39:58.818374 22383 tablet.cc:2369] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:39:58.818406 22574 raft_consensus.cc:2781] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:39:58.818463 22574 raft_consensus.cc:482] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:39:58.818490 22574 raft_consensus.cc:3037] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:58.820283 22574 raft_consensus.cc:504] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.820461 22574 leader_election.cc:288] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [CANDIDATE]: Term 1 election: Requested vote from peers c5c0afc0ef794834a0bc2d7a8eeb9712 (127.7.105.65:39147), cb52b73d10894d23822d7d86fd3af021 (127.7.105.67:40677)
I20260809 00:39:58.820699 22164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0adde499c61443ab91464c68277c7ae" candidate_uuid: "a0697a13d2e14486a82834fb65ea8a18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712"
I20260809 00:39:58.820799 22164 raft_consensus.cc:3037] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:58.822104 22164 raft_consensus.cc:2454] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0697a13d2e14486a82834fb65ea8a18 in term 1.
I20260809 00:39:58.822320 22262 leader_election.cc:302] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0697a13d2e14486a82834fb65ea8a18, c5c0afc0ef794834a0bc2d7a8eeb9712; no voters:
I20260809 00:39:58.822419 22574 raft_consensus.cc:2781] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:39:58.822594 22574 raft_consensus.cc:686] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 1 LEADER]: Becoming Leader. State: Replica: a0697a13d2e14486a82834fb65ea8a18, State: Running, Role: LEADER
I20260809 00:39:58.822705 22574 consensus_queue.cc:237] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } }
I20260809 00:39:58.823662 22022 catalog_manager.cc:5495] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0697a13d2e14486a82834fb65ea8a18 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "a0697a13d2e14486a82834fb65ea8a18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0697a13d2e14486a82834fb65ea8a18" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41567 } health_report { overall_health: HEALTHY } } }
I20260809 00:39:58.827646 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.809568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:46821 (local address 127.7.105.67:40677)
0809 00:39:58.809624 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.809626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.816714 (+ 7088us) server_negotiation.cc:407] Connection header received
0809 00:39:58.816838 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.816839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.816851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.816885 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:39:58.817090 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.817862 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.826464 (+ 8602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.827211 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.827272 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.827283 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.827518 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.827567 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:39:58.827922 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.809441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:40677 (local address 127.7.105.66:46821)
0809 00:39:58.816643 (+ 7202us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.816657 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:39:58.816746 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:39:58.816953 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:39:58.816954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:39:58.816956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:39:58.817004 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.817005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.825070 (+ 8065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.825071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.826328 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:39:58.826332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.827354 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.827355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:39:58.827426 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.827429 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:39:58.827792 (+ 363us) client_negotiation.cc:240] Negotiation successful
0809 00:39:58.827846 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7188,"thread_start_us":543,"threads_started":1}
I20260809 00:39:58.828402 22476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0adde499c61443ab91464c68277c7ae" candidate_uuid: "a0697a13d2e14486a82834fb65ea8a18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb52b73d10894d23822d7d86fd3af021"
I20260809 00:39:58.828449 22477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0adde499c61443ab91464c68277c7ae" candidate_uuid: "a0697a13d2e14486a82834fb65ea8a18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb52b73d10894d23822d7d86fd3af021" is_pre_election: true
I20260809 00:39:58.828613 22476 raft_consensus.cc:3037] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:39:58.829680 22476 raft_consensus.cc:2454] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0697a13d2e14486a82834fb65ea8a18 in term 1.
I20260809 00:39:58.855484 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.852116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.7.105.66:41567)
0809 00:39:58.852176 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:39:58.852179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:39:58.852191 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:39:58.852240 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:39:58.852241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:39:58.852266 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:39:58.852364 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:39:58.852543 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.853261 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.854387 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:39:58.854709 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:39:58.854770 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:39:58.854971 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:39:58.855110 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:39:58.855182 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:39:58.855352 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 00:39:58.855380 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:39:58.858457 22164 raft_consensus.cc:1261] T d0adde499c61443ab91464c68277c7ae P c5c0afc0ef794834a0bc2d7a8eeb9712 [term 1 FOLLOWER]: Refusing update from remote peer a0697a13d2e14486a82834fb65ea8a18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:39:58.858860 22476 raft_consensus.cc:1261] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 1 FOLLOWER]: Refusing update from remote peer a0697a13d2e14486a82834fb65ea8a18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:39:58.859294 22574 consensus_queue.cc:1035] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb52b73d10894d23822d7d86fd3af021" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 40677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:39:58.859760 22574 consensus_queue.cc:1035] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c0afc0ef794834a0bc2d7a8eeb9712" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:39:58.869661 22585 mvcc.cc:205] Tried to move back new op lower bound from 7316422651320152064 to 7316422651178250240. Current Snapshot: MvccSnapshot[applied={T|T < 7316422651320152064 or (T in {7316422651320152064})}]
I20260809 00:39:58.871282 22021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:39:58.871424 22021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:39:58.871594 22586 mvcc.cc:205] Tried to move back new op lower bound from 7316422651320152064 to 7316422651178250240. Current Snapshot: MvccSnapshot[applied={T|T < 7316422651320152064}]
I20260809 00:39:58.872404 22021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4877c5ead874e47a73ab4c8c3059721: Sending DeleteTablet for 3 replicas of tablet d0adde499c61443ab91464c68277c7ae
I20260809 00:39:58.873039 22155 tablet_service.cc:1514] Processing DeleteTablet for tablet d0adde499c61443ab91464c68277c7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:58 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:38738
I20260809 00:39:58.873142 22456 tablet_service.cc:1514] Processing DeleteTablet for tablet d0adde499c61443ab91464c68277c7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:58 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:44098
I20260809 00:39:58.873696 22309 tablet_service.cc:1514] Processing DeleteTablet for tablet d0adde499c61443ab91464c68277c7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:39:58 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:48374
I20260809 00:39:58.874024 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22063
I20260809 00:39:58.879477 22596 tablet_replica.cc:330] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: stopping tablet replica
I20260809 00:39:58.879613 22596 raft_consensus.cc:2229] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:39:58.879752 22596 raft_consensus.cc:2258] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:39:58.879925 22595 tablet_replica.cc:330] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: stopping tablet replica
I20260809 00:39:58.880074 22595 raft_consensus.cc:2229] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:39:58.880260 22595 raft_consensus.cc:2258] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:39:58.880847 22596 ts_tablet_manager.cc:1905] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:39:58.883548 22596 ts_tablet_manager.cc:1918] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:39:58.883641 22596 log.cc:1196] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/d0adde499c61443ab91464c68277c7ae
I20260809 00:39:58.883956 22596 ts_tablet_manager.cc:1939] T d0adde499c61443ab91464c68277c7ae P a0697a13d2e14486a82834fb65ea8a18: Deleting consensus metadata
I20260809 00:39:58.884631 22003 catalog_manager.cc:4841] TS a0697a13d2e14486a82834fb65ea8a18 (127.7.105.66:41567): tablet d0adde499c61443ab91464c68277c7ae (table test-table [id=63b97d163a39441b813790486480d72a]) successfully deleted
I20260809 00:39:58.888186 22595 ts_tablet_manager.cc:1905] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:39:58.890295 22595 ts_tablet_manager.cc:1918] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:39:58.890401 22595 log.cc:1196] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/d0adde499c61443ab91464c68277c7ae
I20260809 00:39:58.890879 22595 ts_tablet_manager.cc:1939] T d0adde499c61443ab91464c68277c7ae P cb52b73d10894d23822d7d86fd3af021: Deleting consensus metadata
I20260809 00:39:58.891388 22003 catalog_manager.cc:4841] TS cb52b73d10894d23822d7d86fd3af021 (127.7.105.67:40677): tablet d0adde499c61443ab91464c68277c7ae (table test-table [id=63b97d163a39441b813790486480d72a]) successfully deleted
W20260809 00:39:58.897161 22006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:39147 (error 108)
I20260809 00:39:58.897925 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22236
I20260809 00:39:58.899806 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:39:58.899684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:39147 (local address 127.0.0.1:38740)
0809 00:39:58.899737 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:39:58.899769 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:39147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:39:58.899921 22006 catalog_manager.cc:4584] TS c5c0afc0ef794834a0bc2d7a8eeb9712 (127.7.105.65:39147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0adde499c61443ab91464c68277c7ae: Network error: Client connection negotiation failed: client connection to 127.7.105.65:39147: connect: Connection refused (error 111)
I20260809 00:39:58.922241 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22385
I20260809 00:39:58.934137 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 21983
I20260809 00:39:58.943001 7589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:39:59.287843 7589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:39:59.287946 7589 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:39:59.633008 7589 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:39:59.633 UTC [20975] LOG: received smart shutdown request
2026-08-09 00:39:59.637 UTC [20975] LOG: background worker "logical replication launcher" (PID 20987) exited with exit code 1
2026-08-09 00:39:59.637 UTC [20979] LOG: shutting down
2026-08-09 00:39:59.818 UTC [20975] LOG: database system is shut down
2026-08-09T00:39:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50911 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:40:00 dist-test-slave-1zqn krb5kdc[22606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:40:00 dist-test-slave-1zqn krb5kdc[22606](info): set up 2 sockets
Aug 09 00:40:00 dist-test-slave-1zqn krb5kdc[22606](info): commencing operation
krb5kdc: starting...
W20260809 00:40:01.073558 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:40:01.208132 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45427
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:01.294327 22621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:01.294602 22621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:01.294646 22621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:01.295847 22621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:01.295922 22621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:01.295945 22621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:01.295966 22621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:01.295993 22621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:01.297768 22621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:45427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:01.298074 22621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:01.298322 22621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:01.303719 22630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:01.303787 22628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:01.303719 22627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:01.304145 22621 server_base.cc:1034] running on GCE node
I20260809 00:40:01.304402 22621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:01.304702 22621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:01.305847 22621 hybrid_clock.cc:648] HybridClock initialized: now 1786236001305818 us; error 43 us; skew 500 ppm
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:01.308821 22621 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:01.309350 22621 webserver.cc:460] Webserver started at http://127.7.105.126:42743/ using document root <none> and password file <none>
I20260809 00:40:01.309590 22621 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:01.309638 22621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:01.309803 22621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:01.310930 22621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "45fcd32e44a8418289ac13a3f00a6194"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "5f5e659e3293c774f7e27c43e2c7817a"
server_key_iv: "fb66ce5d9ee19a187b0a3dbb020ff875"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.311326 22621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "45fcd32e44a8418289ac13a3f00a6194"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "5f5e659e3293c774f7e27c43e2c7817a"
server_key_iv: "fb66ce5d9ee19a187b0a3dbb020ff875"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.312862 22621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 00:40:01.313956 22639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:01.314189 22621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:01.314311 22621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "45fcd32e44a8418289ac13a3f00a6194"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "5f5e659e3293c774f7e27c43e2c7817a"
server_key_iv: "fb66ce5d9ee19a187b0a3dbb020ff875"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.314400 22621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:01.335711 22621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:01.337963 22621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:01.338095 22621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:01.342274 22621 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:45427
I20260809 00:40:01.342329 22704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:45427 every 8 connection(s)
I20260809 00:40:01.342713 22621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:01.343693 22705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:01.344592 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22621
I20260809 00:40:01.344689 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:40:01.344894 7589 external_mini_cluster.cc:1428] Setting key 75744fb418b9ed5eddc85669c8edab50
I20260809 00:40:01.345485 22705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.125@KRBTEST.COM; defaulting to no policy
I20260809 00:40:01.355583 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.346232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45427 (local address 127.0.0.1:52910)
0809 00:40:01.346422 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.346466 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.346942 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.347265 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.347268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.347531 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.347691 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.347695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.348874 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.348875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.349565 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.349566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.350107 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.350108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.350175 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.350784 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.350793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.352270 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.354118 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.354125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.354128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.354324 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.354618 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.354619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.354620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.354749 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.355107 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.355113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.355174 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.355251 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.355489 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":92,"thread_start_us":97,"threads_started":1}
I20260809 00:40:01.355619 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.346255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.7.105.126:45427)
0809 00:40:01.346479 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.346484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.346577 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:40:01.347108 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.347110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.347134 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.347249 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.347791 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.348782 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.349671 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.349996 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.350118 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.352373 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.352380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.352381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.352388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.354007 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.354419 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.354422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.354423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.354529 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.354827 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.354827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.354827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.355040 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.355107 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.355290 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.355404 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
I20260809 00:40:01.356516 22705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.356890 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.356737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58098)
0809 00:40:01.356809 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.356864 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 00:40:01.357033 22643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
I20260809 00:40:01.357503 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.357367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58114)
0809 00:40:01.357430 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.357478 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.125@KRBTEST.COM" created.
W20260809 00:40:01.357661 22705 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:39231: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
I20260809 00:40:01.375200 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:39231
--webserver_interface=127.7.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:01.406893 22705 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } attempt: 1
I20260809 00:40:01.407461 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.407321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58122)
0809 00:40:01.407387 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.407431 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:40:01.407904 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.407790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58128)
0809 00:40:01.407849 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.407884 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 00:40:01.408010 22705 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:39231: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
W20260809 00:40:01.460923 22714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:01.461141 22714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:01.461164 22714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:01.462321 22714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:01.462368 22714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:01.462381 22714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:01.462394 22714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:01.462447 22714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:01.463994 22714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:39231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:01.464308 22714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:01.464565 22714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:01.469637 22719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:01.469643 22720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:01.469659 22722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:01.469870 22714 server_base.cc:1034] running on GCE node
I20260809 00:40:01.470234 22714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:01.470561 22714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:01.471729 22714 hybrid_clock.cc:648] HybridClock initialized: now 1786236001471704 us; error 43 us; skew 500 ppm
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:01.474442 22714 init.cc:377] Logged in from keytab as kudu/127.7.105.125@KRBTEST.COM (short username kudu)
I20260809 00:40:01.474946 22714 webserver.cc:460] Webserver started at http://127.7.105.125:41821/ using document root <none> and password file <none>
I20260809 00:40:01.475231 22714 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:01.475306 22714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:01.475458 22714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:01.476146 22705 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } attempt: 2
I20260809 00:40:01.476538 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.476411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58136)
0809 00:40:01.476473 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.476508 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:40:01.476688 22714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data/instance:
uuid: "615e4cd58f6c4179bcaf48d250f49d49"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "4950ef48bc24227446c8571b55e691fe"
server_key_iv: "112ce7e410dd1d4edcd337fd2b7a9fe8"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.476902 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.476787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58144)
0809 00:40:01.476833 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.476859 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:40:01.476986 22705 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:39231: Network error: Client connection negotiation failed: client connection to 127.7.105.125:39231: connect: Connection refused (error 111)
I20260809 00:40:01.477104 22714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal/instance:
uuid: "615e4cd58f6c4179bcaf48d250f49d49"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "4950ef48bc24227446c8571b55e691fe"
server_key_iv: "112ce7e410dd1d4edcd337fd2b7a9fe8"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.478495 22714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:40:01.479333 22730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:01.479503 22714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:01.479604 22714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "615e4cd58f6c4179bcaf48d250f49d49"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "4950ef48bc24227446c8571b55e691fe"
server_key_iv: "112ce7e410dd1d4edcd337fd2b7a9fe8"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.479662 22714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:01.492247 22714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:01.494675 22714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:01.494801 22714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:01.499038 22714 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.125:39231
I20260809 00:40:01.499118 22792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.125:39231 every 8 connection(s)
I20260809 00:40:01.499490 22714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
I20260809 00:40:01.500514 22793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:01.501137 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22714
I20260809 00:40:01.501217 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-1/wal/instance
I20260809 00:40:01.501412 7589 external_mini_cluster.cc:1428] Setting key 637ac562960e085e6ce27d317fccbbd4
I20260809 00:40:01.502295 22793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.124@KRBTEST.COM; defaulting to no policy
I20260809 00:40:01.513896 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.503119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.7.105.126:45427)
0809 00:40:01.503182 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.503185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.503483 (+ 298us) server_negotiation.cc:407] Connection header received
0809 00:40:01.504328 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.504329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.504340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.504387 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.505299 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.505814 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.507334 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.507702 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.507792 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.510636 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.510637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.510637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.510645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.512256 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.512795 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.512796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.512796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.512871 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.513197 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.513199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.513199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.513353 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.513419 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.513751 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.513826 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:40:01.514133 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.503104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45427 (local address 127.0.0.1:52924)
0809 00:40:01.503319 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.503373 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.504219 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.504466 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.504470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.504872 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.505190 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.505196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.505931 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.505933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.507242 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.507245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.507817 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.507818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.508013 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.508688 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.508697 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.510528 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.512427 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.512436 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.512445 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.512690 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.512975 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.512976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.512977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.513103 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.513451 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.513458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.513616 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.513709 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.513841 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.124@KRBTEST.COM" created.
I20260809 00:40:01.514812 22793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:40:01.522841 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.515105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58156)
0809 00:40:01.515160 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.515173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.515217 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.515477 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.515478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.515621 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.515666 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.515667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.516620 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.516621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.517194 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.517196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.517726 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.517728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.517817 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.518175 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.518181 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.519840 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.521651 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.521652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.521652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.521868 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.522080 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.522081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.522081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.522155 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.522445 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.522446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.522500 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.522588 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.522696 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:40:01.522862 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.515214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.7.105.125:39231)
0809 00:40:01.515356 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.515361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.515375 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:40:01.515390 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.515391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.515401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.515476 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.515743 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.516505 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.517283 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.517601 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.517764 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.519925 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.519933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.519933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.519939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.521560 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.521953 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.521956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.521956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.522009 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.522226 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.522227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.522227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.522362 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.522426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.522624 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.522757 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260809 00:40:01.523535 22793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.523922 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.523800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42244)
0809 00:40:01.523848 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.523893 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:40:01.524075 22731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
I20260809 00:40:01.524536 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.524410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42260)
0809 00:40:01.524486 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.524514 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 00:40:01.524641 22793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33189: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
I20260809 00:40:01.531422 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:33189
--webserver_interface=127.7.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:01.568167 22705 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } attempt: 3
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:40:01.573874 22793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } attempt: 1
I20260809 00:40:01.574321 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.574169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42264)
0809 00:40:01.574238 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.574282 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:40:01.574806 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.574680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42272)
0809 00:40:01.574737 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.574771 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 00:40:01.574965 22793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33189: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
I20260809 00:40:01.576193 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.568606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58160)
0809 00:40:01.568666 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.568687 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.568755 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.568913 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.568915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.569101 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.569173 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.569176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.569895 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.569896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.570424 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.570426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.570824 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.570825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.570911 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.571440 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.571445 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.573133 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.575068 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.575069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.575069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.575244 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.575442 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.575443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.575444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.575534 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.575788 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.575789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.575848 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.575938 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.576078 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:40:01.576196 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.568694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.7.105.125:39231)
0809 00:40:01.568772 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.568774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.568785 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:40:01.568819 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.568820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.568830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.568905 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.569271 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.569793 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.570516 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.570733 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.570823 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.573231 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.573233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.573233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.573240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.574963 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.575311 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.575312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.575313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.575358 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.575616 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.575616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.575617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.575720 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.575771 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.575954 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.576012 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:40:01.576699 22705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.577071 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.576950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42278)
0809 00:40:01.577005 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.577042 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:40:01.577493 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.577398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42288)
0809 00:40:01.577442 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.577472 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:40:01.577600 22705 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33189: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
I20260809 00:40:01.608762 22705 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } attempt: 1
I20260809 00:40:01.609452 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.609252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42290)
0809 00:40:01.609333 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.609404 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 00:40:01.609876 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.609765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42302)
0809 00:40:01.609815 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.609847 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:40:01.609992 22705 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33189: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
W20260809 00:40:01.628439 22801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:01.628661 22801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:01.628686 22801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:01.629767 22801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:01.629817 22801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:01.629833 22801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:01.629848 22801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:01.629863 22801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:01.631376 22801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:45427,127.7.105.125:39231,127.7.105.124:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:01.631592 22801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:01.631852 22801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:01.637418 22807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:01.637509 22808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:01.637493 22801 server_base.cc:1034] running on GCE node
W20260809 00:40:01.637418 22810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:01.637928 22801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:01.638198 22801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:01.639348 22801 hybrid_clock.cc:648] HybridClock initialized: now 1786236001639338 us; error 38 us; skew 500 ppm
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:01.642120 22801 init.cc:377] Logged in from keytab as kudu/127.7.105.124@KRBTEST.COM (short username kudu)
I20260809 00:40:01.642654 22801 webserver.cc:460] Webserver started at http://127.7.105.124:39853/ using document root <none> and password file <none>
I20260809 00:40:01.642895 22801 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:01.642951 22801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:01.643049 22801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:01.643136 22793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } attempt: 2
I20260809 00:40:01.643563 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.643430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42306)
0809 00:40:01.643494 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.643531 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:40:01.643958 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.643874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42310)
0809 00:40:01.643915 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.643941 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:40:01.644091 22793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33189: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33189: connect: Connection refused (error 111)
I20260809 00:40:01.644563 22801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data/instance:
uuid: "27ea4826de8d4e24852f11159e00a38e"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "a586ae7b53deed1892f24e06e19fb99d"
server_key_iv: "5e53f6870d38ca3a729b5b86f54f671b"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.644896 22801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal/instance:
uuid: "27ea4826de8d4e24852f11159e00a38e"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "a586ae7b53deed1892f24e06e19fb99d"
server_key_iv: "5e53f6870d38ca3a729b5b86f54f671b"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.646382 22801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:40:01.647234 22818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:01.647442 22801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:01.647536 22801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "27ea4826de8d4e24852f11159e00a38e"
format_stamp: "Formatted at 2026-08-09 00:40:01 on dist-test-slave-1zqn"
server_key: "a586ae7b53deed1892f24e06e19fb99d"
server_key_iv: "5e53f6870d38ca3a729b5b86f54f671b"
server_key_version: "encryptionkey@0"
I20260809 00:40:01.647600 22801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:01.656021 22801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:01.658394 22801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:01.658536 22801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:01.663041 22801 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.124:33189
I20260809 00:40:01.663072 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.124:33189 every 8 connection(s)
I20260809 00:40:01.663470 22801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
I20260809 00:40:01.664489 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:01.666129 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.667609 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22801
I20260809 00:40:01.667718 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235843038573-7589-0/minicluster-data/master-2/wal/instance
I20260809 00:40:01.667912 7589 external_mini_cluster.cc:1428] Setting key 8fac845179f4c732b8d8642ccbb593b7
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
I20260809 00:40:01.677289 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.666926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.7.105.126:45427)
0809 00:40:01.667007 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.667010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.667338 (+ 328us) server_negotiation.cc:407] Connection header received
0809 00:40:01.668042 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.668044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.668077 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.668166 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.669091 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.669887 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.671198 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.671516 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.671612 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.674261 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.674263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.674263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.674270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.675761 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.676280 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.676281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.676282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.676349 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.676598 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.676599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.676600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.676750 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.676829 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.677093 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.677187 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:40:01.677466 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.667005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:45427 (local address 127.0.0.1:52926)
0809 00:40:01.667190 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.667228 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.667882 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.668245 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.668248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.668667 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.668969 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.668975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.669965 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.669966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.671090 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.671092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.671640 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.671641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.671758 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.672365 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.672374 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.674164 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.675829 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.675837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.675839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.676200 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.676426 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.676426 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.676427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.676528 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.676823 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.676828 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.676943 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.677061 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.677216 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260809 00:40:01.678138 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.678360 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.668418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.7.105.124:33189)
0809 00:40:01.668766 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.668771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.668789 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:40:01.668811 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.668813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.668832 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.668913 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.669763 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.670987 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.672011 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.672403 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.672504 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.675013 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.675021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.675022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.675029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.676923 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.677274 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.677278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.677278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.677363 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.677600 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.677600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.677601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.677761 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.677910 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.678019 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.678103 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":97,"threads_started":1}
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:01.685137 22705 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } attempt: 2
I20260809 00:40:01.685225 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.678529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.7.105.125:39231)
0809 00:40:01.678574 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.678576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.678645 (+ 69us) server_negotiation.cc:407] Connection header received
0809 00:40:01.678891 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.678892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.678902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.678975 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.679271 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.679736 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.680473 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.680663 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.680748 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.682528 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.682529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.682529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.682537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.684098 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.684457 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.684458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.684458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.684514 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.684728 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.684729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.684730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.684844 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.684926 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.685067 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.685142 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:40:01.685662 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.678444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:39231 (local address 127.0.0.1:58164)
0809 00:40:01.678593 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.678602 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.678658 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.678991 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.678992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.679123 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.679197 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.679199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.679825 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.679825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.680383 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.680384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.680745 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.680746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.680816 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.681211 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.681216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.682458 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.684185 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.684187 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.684187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.684356 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.684596 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.684597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.684598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.684654 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.684911 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.684912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.684955 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.685040 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.685546 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 00:40:01.686021 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:40:01.686965 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.679242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.7.105.126:45427)
0809 00:40:01.679281 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.679283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.679291 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:01.679313 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.679314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.679322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.679377 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.679850 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.680361 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.681040 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.681324 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.681398 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.683745 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.683746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.683746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.683753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.685164 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.685935 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.685936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.685936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.686010 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.686232 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.686233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.686233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.686377 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.686573 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.686787 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.686863 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:40:01.693804 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.685451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.7.105.124:33189)
0809 00:40:01.685488 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.685490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.685499 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:40:01.685519 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.685520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.685530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.685899 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.685942 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.686627 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.687526 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.687827 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.687890 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.689758 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.689759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.689759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.689768 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.692484 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.692890 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.692892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.692892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.692976 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.693212 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.693213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.693214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.693409 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.693472 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.693633 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.693708 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:40:01.693804 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.685376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42318)
0809 00:40:01.685424 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.685433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.685472 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.685621 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.685622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.685787 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.685828 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.685830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.686685 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.686686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.687434 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.687437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.687887 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.687888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.687949 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.688352 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.688358 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.689668 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.692572 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.692573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.692573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.692798 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.693056 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.693057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.693058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.693132 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.693465 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.693465 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.693512 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.693607 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.693708 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:40:01.693994 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.686276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42334)
0809 00:40:01.686437 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.686447 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.686481 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.686626 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.686627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.686778 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.686813 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.686815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.687570 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.687570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.688468 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.688471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.688876 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.688876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.688920 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.689307 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.689312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.690721 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.692872 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.692873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.692873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.693097 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.693308 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.693308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.693309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.693360 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.693664 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.693665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.693707 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.693770 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.693870 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260809 00:40:01.694290 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.686306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.7.105.124:33189)
0809 00:40:01.686478 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.686480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.686488 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:01.686531 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.686531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.686540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.686608 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.686904 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.687468 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.688560 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.688756 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.688834 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.690841 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.690842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.690842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.690848 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.692798 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.693174 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.693176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.693176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.693236 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.693443 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.693444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.693445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.693575 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.693640 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.694069 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.694147 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260809 00:40:01.695989 22705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: Bootstrap starting.
I20260809 00:40:01.696535 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.688019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.7.105.125:39231)
0809 00:40:01.688089 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.688091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.688103 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:01.688126 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.688127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.688136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.688193 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.688807 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.689546 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.690508 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.690713 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.690788 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.693485 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.693486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.693486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.693492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.695257 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.695582 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.695583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.695584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.695645 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.695835 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.695836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.695836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.695973 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.696199 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.696248 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.696322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:40:01.696888 22705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:01.697191 22705 log.cc:824] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:01.698197 22705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: No bootstrap required, opened a new log
I20260809 00:40:01.698583 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:40:01.698927 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: Bootstrap starting.
I20260809 00:40:01.699555 22875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:01.699935 22875 log.cc:824] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:01.700513 22705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.700760 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: No bootstrap required, opened a new log
I20260809 00:40:01.700810 22705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:01.700894 22705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fcd32e44a8418289ac13a3f00a6194, State: Initialized, Role: FOLLOWER
I20260809 00:40:01.701072 22705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.701375 22705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:01.703135 22875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.703285 22875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:01.703322 22875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ea4826de8d4e24852f11159e00a38e, State: Initialized, Role: FOLLOWER
I20260809 00:40:01.703447 22875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.703753 22875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:01.705550 22621 master_runner.cc:186] Error getting master registration for 127.7.105.125:39231: OK, instance_id { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" instance_seqno: 1786236001497876 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:40:01.707948 22621 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:40:01.708267 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.700521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.7.105.125:39231)
0809 00:40:01.700570 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.700572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.700793 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:40:01.700887 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.700888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.700899 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.700938 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.701590 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.702274 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.703382 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.703678 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.703715 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.705058 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.705060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.705060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.705068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.706856 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.707279 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.707280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.707281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.707340 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.707585 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.707586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.707586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.707724 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.707804 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.708030 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.708154 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:40:01.708557 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.708776 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:40:01.703512 22902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:40:01.709388 22902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:40:01.709830 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.700286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.7.105.126:45427)
0809 00:40:01.700331 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.700333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.700472 (+ 139us) server_negotiation.cc:407] Connection header received
0809 00:40:01.700575 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.700576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.700587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.700619 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.701447 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.702212 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.703284 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.703563 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.703603 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.706214 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.706216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.706216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.706224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.707921 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.708438 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.708439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.708440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.708537 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.708780 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.708781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.708782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.708964 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.709094 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.709662 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.709747 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:40:01.710963 22918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:40:01.711071 22918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:40:01.711431 22906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.711602 22906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:40:01.711031 22801 master_runner.cc:186] Error getting master registration for 127.7.105.125:39231: OK, instance_id { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" instance_seqno: 1786236001497876 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:40:01.713752 22801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:40:01.715961 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.702591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.7.105.124:33189)
0809 00:40:01.704003 (+ 1412us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.704005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.709016 (+ 5011us) server_negotiation.cc:407] Connection header received
0809 00:40:01.709044 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.709045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.709057 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.709104 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.709675 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.710413 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.711317 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.711629 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.711688 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.713156 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.713158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.713158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.713165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.714718 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.715101 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.715102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.715103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.715158 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.715369 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.715370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.715370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.715487 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.715608 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.715705 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.715783 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388}
I20260809 00:40:01.735260 22793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } attempt: 3
Aug 09 00:40:01 dist-test-slave-1zqn krb5kdc[22606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236001, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
I20260809 00:40:01.742725 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.735582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33189 (local address 127.0.0.1:42354)
0809 00:40:01.735638 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:01.735650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:01.735693 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:01.735842 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:01.735844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:01.736019 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:01.736127 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.736130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.736794 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.736796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.737311 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:01.737312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.737684 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.737685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:01.737751 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.738228 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:01.738238 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:01.739491 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:01.741465 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.741466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.741467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.741668 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.741926 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:01.741927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:01.741927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:01.742002 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:01.742340 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:01.742342 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:01.742419 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 00:40:01.742472 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:40:01.742592 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:40:01.743505 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:01.735644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.7.105.124:33189)
0809 00:40:01.735684 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:01.735686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:01.735694 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:01.735748 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:01.735749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:01.735759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:01.735834 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:01.736206 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.736698 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.737399 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:01.737595 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:01.737682 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:01.739599 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:01.739600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:01.739601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:01.739614 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:01.741343 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.741766 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.741767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.741768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.741844 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:01.742090 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:01.742091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:01.742091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:01.742244 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:01.743246 (+ 1002us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:01.743343 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:40:01.743424 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:40:01.745411 22793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: Bootstrap starting.
I20260809 00:40:01.746097 22793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:01.746426 22793 log.cc:824] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:01.747241 22793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: No bootstrap required, opened a new log
I20260809 00:40:01.748781 22793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.748925 22793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:01.748993 22793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615e4cd58f6c4179bcaf48d250f49d49, State: Initialized, Role: FOLLOWER
I20260809 00:40:01.749152 22793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.749492 22923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.749565 22793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:01.749637 22923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:40:01.752548 22934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:40:01.752645 22934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:40:01.754669 22714 master_runner.cc:263] No leader master found from master 615e4cd58f6c4179bcaf48d250f49d49
I20260809 00:40:01.754719 22714 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:40:01.818149 22923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:01.818305 22923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.818562 22923 leader_election.cc:288] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45fcd32e44a8418289ac13a3f00a6194 (127.7.105.126:45427), 27ea4826de8d4e24852f11159e00a38e (127.7.105.124:33189)
I20260809 00:40:01.819149 22847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "615e4cd58f6c4179bcaf48d250f49d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27ea4826de8d4e24852f11159e00a38e" is_pre_election: true
I20260809 00:40:01.819321 22847 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615e4cd58f6c4179bcaf48d250f49d49 in term 0.
I20260809 00:40:01.819660 22731 leader_election.cc:302] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27ea4826de8d4e24852f11159e00a38e, 615e4cd58f6c4179bcaf48d250f49d49; no voters:
I20260809 00:40:01.819847 22923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:40:01.819900 22923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:40:01.819924 22923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:01.820927 22923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.821061 22923 leader_election.cc:288] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [CANDIDATE]: Term 1 election: Requested vote from peers 45fcd32e44a8418289ac13a3f00a6194 (127.7.105.126:45427), 27ea4826de8d4e24852f11159e00a38e (127.7.105.124:33189)
I20260809 00:40:01.821367 22847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "615e4cd58f6c4179bcaf48d250f49d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27ea4826de8d4e24852f11159e00a38e"
I20260809 00:40:01.821450 22847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:01.822412 22847 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615e4cd58f6c4179bcaf48d250f49d49 in term 1.
I20260809 00:40:01.822662 22731 leader_election.cc:302] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27ea4826de8d4e24852f11159e00a38e, 615e4cd58f6c4179bcaf48d250f49d49; no voters:
I20260809 00:40:01.823161 22675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "615e4cd58f6c4179bcaf48d250f49d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fcd32e44a8418289ac13a3f00a6194" is_pre_election: true
I20260809 00:40:01.823330 22675 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615e4cd58f6c4179bcaf48d250f49d49 in term 0.
I20260809 00:40:01.823462 22674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "615e4cd58f6c4179bcaf48d250f49d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fcd32e44a8418289ac13a3f00a6194"
I20260809 00:40:01.823527 22674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:01.824622 22674 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615e4cd58f6c4179bcaf48d250f49d49 in term 1.
I20260809 00:40:01.825886 22923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:01.826064 22923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [term 1 LEADER]: Becoming Leader. State: Replica: 615e4cd58f6c4179bcaf48d250f49d49, State: Running, Role: LEADER
I20260809 00:40:01.826160 22923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } }
I20260809 00:40:01.826565 22923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 615e4cd58f6c4179bcaf48d250f49d49. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.826637 22923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:01.826999 22938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:01.827270 22938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:01.829587 22847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [term 1 FOLLOWER]: Refusing update from remote peer 615e4cd58f6c4179bcaf48d250f49d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:01.829882 22674 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [term 1 FOLLOWER]: Refusing update from remote peer 615e4cd58f6c4179bcaf48d250f49d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:01.830421 22923 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:01.830571 22923 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:01.832017 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 615e4cd58f6c4179bcaf48d250f49d49. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.832144 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:40:01.833312 22906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 615e4cd58f6c4179bcaf48d250f49d49. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.833434 22906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:40:01.835605 22923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.835697 22923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:01.836258 22906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.836344 22906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:40:01.836625 22940 mvcc.cc:205] Tried to move back new op lower bound from 7316422663490793472 to 7316422663480504320. Current Snapshot: MvccSnapshot[applied={T|T < 7316422663490793472}]
I20260809 00:40:01.836855 22943 mvcc.cc:205] Tried to move back new op lower bound from 7316422663490793472 to 7316422663480504320. Current Snapshot: MvccSnapshot[applied={T|T < 7316422663490793472}]
I20260809 00:40:01.837325 22923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.837405 22923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:01.837834 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "615e4cd58f6c4179bcaf48d250f49d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fcd32e44a8418289ac13a3f00a6194" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 45427 } } peers { permanent_uuid: "615e4cd58f6c4179bcaf48d250f49d49" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 39231 } } peers { permanent_uuid: "27ea4826de8d4e24852f11159e00a38e" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33189 } } }
I20260809 00:40:01.837924 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:40:01.838678 22941 mvcc.cc:205] Tried to move back new op lower bound from 7316422663490793472 to 7316422663480504320. Current Snapshot: MvccSnapshot[applied={T|T < 7316422663490793472}]
I20260809 00:40:01.839644 22938 catalog_manager.cc:1347] Generated new cluster ID: 54737b219b784121938689cab1cf51e8
I20260809 00:40:01.839692 22938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:40:01.863570 22938 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:40:01.865408 22938 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:01.877163 22938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 615e4cd58f6c4179bcaf48d250f49d49: Generated new TSK 0
I20260809 00:40:01.877460 22938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:40:02.709934 22902 catalog_manager.cc:1259] Loaded cluster ID: 54737b219b784121938689cab1cf51e8
I20260809 00:40:02.710031 22902 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: loading cluster ID for follower catalog manager: success
I20260809 00:40:02.711063 22902 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: acquiring CA information for follower catalog manager: success
I20260809 00:40:02.711428 22902 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45fcd32e44a8418289ac13a3f00a6194: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:40:02.711509 22918 catalog_manager.cc:1259] Loaded cluster ID: 54737b219b784121938689cab1cf51e8
I20260809 00:40:02.711632 22918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: loading cluster ID for follower catalog manager: success
I20260809 00:40:02.712476 22918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: acquiring CA information for follower catalog manager: success
I20260809 00:40:02.712776 22918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 27ea4826de8d4e24852f11159e00a38e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:40:02.932647 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22621
I20260809 00:40:02.941191 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22714
I20260809 00:40:02.948875 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22801
2026-08-09T00:40:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2999 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:40:03 dist-test-slave-1zqn krb5kdc[22956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:40:03 dist-test-slave-1zqn krb5kdc[22956](info): set up 2 sockets
Aug 09 00:40:03 dist-test-slave-1zqn krb5kdc[22956](info): commencing operation
krb5kdc: starting...
W20260809 00:40:04.057803 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[22956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:40:04.191322 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43073
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:04.280088 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:04.280305 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:04.280328 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:04.281418 22969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:04.281453 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:04.281468 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:04.281481 22969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:04.281492 22969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:04.282933 22969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:04.283141 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:04.283320 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:04.288368 22975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:04.288367 22974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:04.288342 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:04.288561 22969 server_base.cc:1034] running on GCE node
I20260809 00:40:04.288861 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:04.289196 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:04.290349 22969 hybrid_clock.cc:648] HybridClock initialized: now 1786236004290329 us; error 43 us; skew 500 ppm
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[22956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:04.293186 22969 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:04.293646 22969 webserver.cc:460] Webserver started at http://127.7.105.126:46007/ using document root <none> and password file <none>
I20260809 00:40:04.293893 22969 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:04.293965 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:04.294122 22969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:04.295243 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "3f65e834ebb04e678169a1b3cdef6f1d"
format_stamp: "Formatted at 2026-08-09 00:40:04 on dist-test-slave-1zqn"
server_key: "e23e4aad7955742ac2c5dcf8310c3035"
server_key_iv: "b37d8e10d0aef6787f1d9382e3c92af9"
server_key_version: "encryptionkey@0"
I20260809 00:40:04.295626 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "3f65e834ebb04e678169a1b3cdef6f1d"
format_stamp: "Formatted at 2026-08-09 00:40:04 on dist-test-slave-1zqn"
server_key: "e23e4aad7955742ac2c5dcf8310c3035"
server_key_iv: "b37d8e10d0aef6787f1d9382e3c92af9"
server_key_version: "encryptionkey@0"
I20260809 00:40:04.297125 22969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:40:04.297873 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:04.298070 22969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:40:04.298195 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "3f65e834ebb04e678169a1b3cdef6f1d"
format_stamp: "Formatted at 2026-08-09 00:40:04 on dist-test-slave-1zqn"
server_key: "e23e4aad7955742ac2c5dcf8310c3035"
server_key_iv: "b37d8e10d0aef6787f1d9382e3c92af9"
server_key_version: "encryptionkey@0"
I20260809 00:40:04.298272 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:04.316777 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:04.319247 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:04.319414 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:04.323665 22969 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:43073
I20260809 00:40:04.323734 23043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:43073 every 8 connection(s)
I20260809 00:40:04.324144 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:04.325135 23044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:04.327394 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 22969
I20260809 00:40:04.327507 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:40:04.327581 23044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d: Bootstrap starting.
I20260809 00:40:04.327711 7589 external_mini_cluster.cc:1428] Setting key c8146087537f5e00e8eff6d21b261a1f
I20260809 00:40:04.328334 23044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:04.328680 23044 log.cc:824] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:04.329380 23044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d: No bootstrap required, opened a new log
I20260809 00:40:04.331135 23044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } }
I20260809 00:40:04.331293 23044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:04.331341 23044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f65e834ebb04e678169a1b3cdef6f1d, State: Initialized, Role: FOLLOWER
I20260809 00:40:04.331518 23044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } }
I20260809 00:40:04.331625 23044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:04.331670 23044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:04.331720 23044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:04.332306 23044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } }
I20260809 00:40:04.332424 23044 leader_election.cc:302] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f65e834ebb04e678169a1b3cdef6f1d; no voters:
I20260809 00:40:04.332623 23044 leader_election.cc:288] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:40:04.332648 23049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:04.332793 23049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [term 1 LEADER]: Becoming Leader. State: Replica: 3f65e834ebb04e678169a1b3cdef6f1d, State: Running, Role: LEADER
I20260809 00:40:04.332895 23044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:04.332902 23049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } }
I20260809 00:40:04.333312 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f65e834ebb04e678169a1b3cdef6f1d. Latest consensus state: current_term: 1 leader_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } } }
I20260809 00:40:04.333293 23050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f65e834ebb04e678169a1b3cdef6f1d" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43073 } } }
I20260809 00:40:04.333513 23050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:04.333495 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:04.334465 23058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:04.334978 23058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[22956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:04.336529 23058 catalog_manager.cc:1347] Generated new cluster ID: 4d166b1594c7470eb667e19a1890db8b
I20260809 00:40:04.336585 23058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:40:04.343264 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:04.328255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.7.105.126:43073)
0809 00:40:04.328455 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:04.328465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:04.328499 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:40:04.328991 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:04.328999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:04.329048 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:04.329203 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:04.329899 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:04.330750 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:04.331590 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:04.332180 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:04.332298 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:04.335817 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:04.335821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:04.335821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:04.335834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:04.341460 (+ 5626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:04.341900 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:04.341904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:04.341905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:04.342030 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:04.342246 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:04.342247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:04.342247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:04.342578 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:04.342745 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:04.342867 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 00:40:04.343022 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260809 00:40:04.344295 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:40:04.355381 23058 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:40:04.355962 23058 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:04.358577 23058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f65e834ebb04e678169a1b3cdef6f1d: Generated new TSK 0
I20260809 00:40:04.358776 23058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:40:04.398325 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 22969
2026-08-09T00:40:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1447 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[23072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[23072](info): set up 2 sockets
Aug 09 00:40:04 dist-test-slave-1zqn krb5kdc[23072](info): commencing operation
krb5kdc: starting...
W20260809 00:40:05.499424 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:40:05 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:40:05.623302 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:05.706573 23087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:05.706774 23087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:05.706796 23087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:05.708025 23087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:05.708101 23087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:05.708127 23087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:05.708153 23087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:05.708171 23087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:05.709602 23087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:05.709791 23087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:05.709954 23087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:05.715000 23095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:05.714947 23092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:05.714947 23093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:05.715077 23087 server_base.cc:1034] running on GCE node
I20260809 00:40:05.715459 23087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:05.715793 23087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:05.717131 23087 hybrid_clock.cc:648] HybridClock initialized: now 1786236005717104 us; error 38 us; skew 500 ppm
Aug 09 00:40:05 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:05.724462 23087 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:05.724978 23087 webserver.cc:460] Webserver started at http://127.7.105.126:33923/ using document root <none> and password file <none>
I20260809 00:40:05.725209 23087 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:05.725276 23087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:05.725427 23087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:05.726560 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "86d79253f6654d3fbc1c7ccfac7641c2"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "da727633914708bd1802ea47f0656d7f"
server_key_iv: "481076a40d0b5894fbf454cbf13a7ba8"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.726969 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "86d79253f6654d3fbc1c7ccfac7641c2"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "da727633914708bd1802ea47f0656d7f"
server_key_iv: "481076a40d0b5894fbf454cbf13a7ba8"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.728451 23087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:40:05.729301 23103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:05.729513 23087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:05.729626 23087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "86d79253f6654d3fbc1c7ccfac7641c2"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "da727633914708bd1802ea47f0656d7f"
server_key_iv: "481076a40d0b5894fbf454cbf13a7ba8"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.729702 23087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:05.741943 23087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:05.743940 23087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:05.744170 23087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:05.748551 23087 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44961
I20260809 00:40:05.748605 23160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44961 every 8 connection(s)
I20260809 00:40:05.748956 23087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:05.749982 23161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:05.752314 23161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap starting.
I20260809 00:40:05.752911 23161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:05.753221 23161 log.cc:824] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:05.753966 23161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: No bootstrap required, opened a new log
I20260809 00:40:05.755622 23161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:05.755764 23161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:05.755786 23161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Initialized, Role: FOLLOWER
I20260809 00:40:05.755946 23161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:05.756011 23161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:05.756031 23161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:05.756121 23161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:05.756949 23161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:05.757038 23161 leader_election.cc:302] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86d79253f6654d3fbc1c7ccfac7641c2; no voters:
I20260809 00:40:05.757258 23161 leader_election.cc:288] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:40:05.757332 23165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:05.757511 23161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:05.757517 23165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 LEADER]: Becoming Leader. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Running, Role: LEADER
I20260809 00:40:05.757649 23165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:05.758296 23167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d79253f6654d3fbc1c7ccfac7641c2. Latest consensus state: current_term: 1 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:05.758433 23167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:05.758296 23166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:05.758502 23166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:05.758559 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23087
I20260809 00:40:05.758672 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:40:05.758673 23177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:05.758898 7589 external_mini_cluster.cc:1428] Setting key f0585c19bb6d22973228c06dda4f4755
I20260809 00:40:05.759195 23177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:05.761186 23177 catalog_manager.cc:1347] Generated new cluster ID: fa8bb02caf094e37ace5de2150d3bc45
I20260809 00:40:05.761232 23177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:40:05 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:05.779893 23177 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:40:05.780607 23177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:05.782333 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:05.759585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.7.105.126:44961)
0809 00:40:05.759762 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:05.759771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:05.759989 (+ 218us) server_negotiation.cc:407] Connection header received
0809 00:40:05.760151 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:05.760158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:05.760218 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:05.760369 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:05.772258 (+ 11889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.773674 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.774840 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.775458 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.775558 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:05.778480 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:05.778485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:05.778485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:05.778499 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:05.780719 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:05.781100 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:05.781102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:05.781103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:05.781209 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:05.781413 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:05.781414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:05.781414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:05.781681 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:05.781832 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:05.781959 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 00:40:05.782102 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 00:40:05.785630 23177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Generated new TSK 0
I20260809 00:40:05.785808 23177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:40:05.815088 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:05.900363 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:05.900607 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:05.900645 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:05.901794 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:05.901866 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:05.901913 23194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:40:05.903390 23194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:05.903683 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:05.903921 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:05.909896 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:05.909816 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:05.913215 23194 server_base.cc:1034] running on GCE node
W20260809 00:40:05.913468 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:05.913729 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:05.914070 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:05.915328 23194 hybrid_clock.cc:648] HybridClock initialized: now 1786236005915302 us; error 41 us; skew 500 ppm
Aug 09 00:40:05 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:05.918190 23194 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:40:05.918679 23194 webserver.cc:460] Webserver started at http://127.7.105.65:40995/ using document root <none> and password file <none>
I20260809 00:40:05.918960 23194 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:05.919009 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:05.919106 23194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:05.920197 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "584645ed78c649f9a51e34588665574b"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "d48f245ebad1d33579ff45d42ec32c4b"
server_key_iv: "7efc5f7ddf499ea93752b3b5656ecf53"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.920504 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "584645ed78c649f9a51e34588665574b"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "d48f245ebad1d33579ff45d42ec32c4b"
server_key_iv: "7efc5f7ddf499ea93752b3b5656ecf53"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.921790 23194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:05.922477 23214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:05.922663 23194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:40:05.922771 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "584645ed78c649f9a51e34588665574b"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "d48f245ebad1d33579ff45d42ec32c4b"
server_key_iv: "7efc5f7ddf499ea93752b3b5656ecf53"
server_key_version: "encryptionkey@0"
I20260809 00:40:05.922844 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:05.937469 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:05.939265 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:05.939424 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:05.939649 23194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:05.939960 23194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:05.940013 23194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:05.940083 23194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:05.940132 23194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:05.945904 23194 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35877
I20260809 00:40:05.946008 23335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35877 every 8 connection(s)
I20260809 00:40:05.946306 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:40:05 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:05.954159 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23194
I20260809 00:40:05.954296 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:40:05.954499 7589 external_mini_cluster.cc:1428] Setting key fea50e7490fbf91f53d56ffe04e90661
I20260809 00:40:05.955912 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:05.947032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:42681)
0809 00:40:05.947189 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:05.947239 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:05.947750 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:05.948004 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:05.948008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:05.948664 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:05.948854 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:05.948858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.949574 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.949576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:05.950374 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:05.950376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.950773 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.950774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:05.950910 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:05.951456 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:05.951464 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:05.952856 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:05.954452 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:05.954460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:05.954462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:05.954784 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:05.954981 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:05.954982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:05.954983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:05.955101 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:05.955416 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:05.955422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:05.955547 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 00:40:05.955632 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:40:05.955754 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 00:40:05.955930 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:05.947062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:42681 (local address 127.7.105.126:44961)
0809 00:40:05.947137 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:05.947140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:05.947325 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:40:05.947858 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:05.947860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:05.947872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:05.947932 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:05.948964 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.949472 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.950477 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:05.950687 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:05.950763 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:05.952959 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:05.952960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:05.952961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:05.952969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:05.954375 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:05.954845 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:05.954846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:05.954847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:05.954912 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:05.955175 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:05.955176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:05.955176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:05.955339 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:05.955422 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:05.955686 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 00:40:05.955795 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:40:05.956516 23336 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:05.956634 23336 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:05.956815 23336 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:05.957471 23121 ts_manager.cc:194] Registered new tserver with Master: 584645ed78c649f9a51e34588665574b (127.7.105.65:35877)
I20260809 00:40:05.958163 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:42681
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:40:05.980902 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:06.063905 23341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.064173 23341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.064214 23341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.065375 23341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.065447 23341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.065495 23341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:40:06.067035 23341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.067322 23341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.067576 23341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.073060 23346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.073086 23347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.073064 23349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.073098 23341 server_base.cc:1034] running on GCE node
I20260809 00:40:06.073483 23341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.073762 23341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.074908 23341 hybrid_clock.cc:648] HybridClock initialized: now 1786236006074902 us; error 38 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.077462 23341 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:40:06.077901 23341 webserver.cc:460] Webserver started at http://127.7.105.66:39719/ using document root <none> and password file <none>
I20260809 00:40:06.078137 23341 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.078186 23341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.078351 23341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:06.079414 23341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "949919c78932479794df00fa2197fb10"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "9d623781f314866c1bc2c165a08c5e78"
server_key_iv: "5ccad6158889590b6ad7ba7dd3fab1d8"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.079842 23341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "949919c78932479794df00fa2197fb10"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "9d623781f314866c1bc2c165a08c5e78"
server_key_iv: "5ccad6158889590b6ad7ba7dd3fab1d8"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.081297 23341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:06.082077 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.082266 23341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:40:06.082360 23341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "949919c78932479794df00fa2197fb10"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "9d623781f314866c1bc2c165a08c5e78"
server_key_iv: "5ccad6158889590b6ad7ba7dd3fab1d8"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.082429 23341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.103657 23341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.105835 23341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.105985 23341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.106251 23341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:06.106604 23341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:06.106666 23341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.106710 23341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:06.106738 23341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.112687 23341 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35857
I20260809 00:40:06.112743 23478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35857 every 8 connection(s)
I20260809 00:40:06.113135 23341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:40:06.116084 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23341
I20260809 00:40:06.116219 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:40:06.116403 7589 external_mini_cluster.cc:1428] Setting key b7481dabd93eac4631e8eb4f8aa67452
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:06.123951 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.113924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:34237 (local address 127.7.105.126:44961)
0809 00:40:06.113968 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.113970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.114157 (+ 187us) server_negotiation.cc:407] Connection header received
0809 00:40:06.114696 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.114697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.114711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.114769 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.115741 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.116410 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.117679 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.117888 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.117973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.120729 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.120731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.120731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.120740 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.122520 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.122928 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.122929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.122929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.122998 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.123245 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.123246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.123246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.123405 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.123483 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.123746 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.123838 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:40:06.123980 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.113877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:34237)
0809 00:40:06.114026 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.114074 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.114574 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.114844 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.114848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.115434 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.115640 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.115645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.116504 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.116505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.117576 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.117585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.117983 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.117985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.118125 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.118705 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.118715 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.120629 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.122612 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.122619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.122620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.122848 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.123071 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.123072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.123072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.123175 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.123475 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.123480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.123612 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.123695 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.123816 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260809 00:40:06.124565 23479 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:06.124662 23479 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:06.124861 23479 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:06.125290 23121 ts_manager.cc:194] Registered new tserver with Master: 949919c78932479794df00fa2197fb10 (127.7.105.66:35857)
I20260809 00:40:06.125748 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:34237
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:40:06.149387 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:06.236187 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.236393 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.236419 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.237607 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.237654 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.237681 23484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:40:06.239152 23484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.239395 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.239629 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.246640 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.246666 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.246693 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.247113 23484 server_base.cc:1034] running on GCE node
I20260809 00:40:06.247373 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.247674 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.248849 23484 hybrid_clock.cc:648] HybridClock initialized: now 1786236006248810 us; error 61 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.251546 23484 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:40:06.252095 23484 webserver.cc:460] Webserver started at http://127.7.105.67:46045/ using document root <none> and password file <none>
I20260809 00:40:06.252354 23484 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.252439 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.252605 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:06.253784 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "8e903de616f7475ab45b506cb1b1d94a"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "4596cbbd111c1db106b2bc496a2132b9"
server_key_iv: "3c78958f34fb564d28540eddc4525ef3"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.254184 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "8e903de616f7475ab45b506cb1b1d94a"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "4596cbbd111c1db106b2bc496a2132b9"
server_key_iv: "3c78958f34fb564d28540eddc4525ef3"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.255752 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:40:06.256656 23499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.256850 23484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:06.256963 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "8e903de616f7475ab45b506cb1b1d94a"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "4596cbbd111c1db106b2bc496a2132b9"
server_key_iv: "3c78958f34fb564d28540eddc4525ef3"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.257054 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.270143 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.272840 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.273025 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.273272 23484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:06.273629 23484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:06.273689 23484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.273741 23484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:06.273777 23484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.280213 23484 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:39559
I20260809 00:40:06.280279 23622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:39559 every 8 connection(s)
I20260809 00:40:06.280651 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:40:06.285215 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23484
I20260809 00:40:06.285316 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:40:06.285578 7589 external_mini_cluster.cc:1428] Setting key 6fbce1973b36379b2c989663400b1893
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:06.290602 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.281429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36269 (local address 127.7.105.126:44961)
0809 00:40:06.281501 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.281504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.281702 (+ 198us) server_negotiation.cc:407] Connection header received
0809 00:40:06.282175 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.282176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.282187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.282248 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.283450 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.283996 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.285028 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.285414 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.285517 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.287841 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.287843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.287843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.287850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.289476 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.289834 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.289835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.289835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.289890 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.290101 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.290102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.290102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.290208 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.290266 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.290470 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.290534 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:40:06.290846 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.281401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:36269)
0809 00:40:06.281568 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.281609 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.282092 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.282606 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.282610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.283169 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.283354 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.283359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.284132 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.284133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.284925 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.284927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.285528 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.285529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.285699 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.286301 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.286311 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.287757 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.289549 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.289554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.289556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.289767 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.289949 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.289950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.289950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.290044 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.290266 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.290270 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.290376 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.290454 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.290570 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 00:40:06.291438 23623 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:06.291519 23623 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:06.291716 23623 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:06.292188 23121 ts_manager.cc:194] Registered new tserver with Master: 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:06.292558 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:36269
I20260809 00:40:06.297631 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:40:06.305086 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.299113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52888 (local address 127.7.105.126:44961)
0809 00:40:06.299172 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.299174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.299297 (+ 123us) server_negotiation.cc:407] Connection header received
0809 00:40:06.299334 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.299334 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.299343 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.299394 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.299913 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.300502 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.301217 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.301401 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.301466 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.302602 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.302604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.302604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.302610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.304149 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.304438 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.304438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.304439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.304492 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.304648 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.304648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.304648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.304780 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.304850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.304936 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.304992 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:40:06.306941 23121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:40:06.308789 23121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:40:06.315399 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35857 (local address 127.0.0.1:33928)
0809 00:40:06.311210 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.311251 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.311312 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.311731 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.311734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.311736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:06.311823 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.311825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.313204 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.313205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.314383 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.314385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.315082 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.315166 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315180 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.315246 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.315305 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":8,"thread_start_us":73,"threads_started":1}
I20260809 00:40:06.315523 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.7.105.67:39559)
0809 00:40:06.311780 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.311785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.312001 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:40:06.312050 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.312074 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.312091 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.312167 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:06.312275 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.312939 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.314482 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.315176 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315253 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315308 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.315394 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.315434 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260809 00:40:06.315712 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.7.105.65:35877)
0809 00:40:06.311637 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.311641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.311757 (+ 116us) server_negotiation.cc:407] Connection header received
0809 00:40:06.311876 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.311879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.311896 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.311949 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:06.312206 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.313336 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.314435 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.315176 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315254 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315308 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.315566 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.315628 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260809 00:40:06.315722 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.7.105.66:35857)
0809 00:40:06.311441 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.311446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.311458 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:06.311500 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.311502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.311520 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.311625 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:06.311906 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.313099 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.314490 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.314983 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315070 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315134 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.315600 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.315643 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
I20260809 00:40:06.316128 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35877 (local address 127.0.0.1:48280)
0809 00:40:06.311691 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.311702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.311783 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.312048 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.312049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.312072 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:06.312125 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.312126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.313430 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.313431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.314349 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.314351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.315359 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.315425 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315429 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.315479 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.315518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":80,"threads_started":1}
I20260809 00:40:06.316138 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.311492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39559 (local address 127.0.0.1:41636)
0809 00:40:06.311851 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.311862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.311914 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.312165 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.312165 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.312167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:06.312217 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.312219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.313047 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.313048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.314376 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.314379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.315252 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.315253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.315306 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.315309 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.315364 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.315398 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":84,"threads_started":1}
I20260809 00:40:06.316494 23409 tablet_service.cc:1467] Processing CreateTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b (DEFAULT_TABLE table=test-table [id=6e5b163e477d494bab840e782a6ba843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:06.316920 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eb9dfb3a494b2d87af2a38851cff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:06.316931 23266 tablet_service.cc:1467] Processing CreateTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b (DEFAULT_TABLE table=test-table [id=6e5b163e477d494bab840e782a6ba843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:06.317008 23550 tablet_service.cc:1467] Processing CreateTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b (DEFAULT_TABLE table=test-table [id=6e5b163e477d494bab840e782a6ba843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:06.317286 23266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eb9dfb3a494b2d87af2a38851cff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:06.317327 23550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eb9dfb3a494b2d87af2a38851cff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:06.320134 23640 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Bootstrap starting.
I20260809 00:40:06.320513 23641 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Bootstrap starting.
I20260809 00:40:06.320906 23640 tablet_bootstrap.cc:654] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:06.320950 23642 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Bootstrap starting.
I20260809 00:40:06.321144 23641 tablet_bootstrap.cc:654] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:06.321540 23640 log.cc:824] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:06.321651 23642 tablet_bootstrap.cc:654] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:06.321805 23641 log.cc:824] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:06.322273 23642 log.cc:824] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:06.322455 23640 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: No bootstrap required, opened a new log
I20260809 00:40:06.322559 23640 ts_tablet_manager.cc:1397] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:06.322616 23641 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: No bootstrap required, opened a new log
I20260809 00:40:06.322685 23641 ts_tablet_manager.cc:1397] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:06.323043 23642 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: No bootstrap required, opened a new log
I20260809 00:40:06.323115 23642 ts_tablet_manager.cc:1397] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:06.324383 23641 raft_consensus.cc:348] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.324539 23641 raft_consensus.cc:374] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:06.324591 23641 raft_consensus.cc:729] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949919c78932479794df00fa2197fb10, State: Initialized, Role: FOLLOWER
I20260809 00:40:06.324571 23640 raft_consensus.cc:348] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.324747 23640 raft_consensus.cc:374] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:06.324705 23641 consensus_queue.cc:260] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.324793 23640 raft_consensus.cc:729] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584645ed78c649f9a51e34588665574b, State: Initialized, Role: FOLLOWER
I20260809 00:40:06.324895 23640 consensus_queue.cc:260] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.325027 23641 ts_tablet_manager.cc:1428] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:40:06.325186 23640 ts_tablet_manager.cc:1428] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:40:06.325090 23642 raft_consensus.cc:348] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.325248 23642 raft_consensus.cc:374] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:06.325281 23642 raft_consensus.cc:729] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e903de616f7475ab45b506cb1b1d94a, State: Initialized, Role: FOLLOWER
I20260809 00:40:06.325376 23479 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
I20260809 00:40:06.325397 23642 consensus_queue.cc:260] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.325676 23336 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
I20260809 00:40:06.325670 23642 ts_tablet_manager.cc:1428] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:40:06.325717 23623 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
I20260809 00:40:06.348075 23647 raft_consensus.cc:482] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:06.348198 23647 raft_consensus.cc:504] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.348624 23647 leader_election.cc:288] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:06.351366 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.348836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:48047 (local address 127.7.105.66:35857)
0809 00:40:06.348901 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.348906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.348938 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:40:06.348961 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.348962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.348970 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.349038 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:06.349216 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.349709 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.350632 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.351005 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.351095 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.351106 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.351264 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.351301 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:40:06.351366 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.348769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35857 (local address 127.7.105.65:48047)
0809 00:40:06.348840 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.348853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.348898 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.349047 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.349048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.349051 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:06.349141 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.349143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.349796 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.349797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.350523 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.350525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.351097 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.351098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.351182 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.351185 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.351245 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.351287 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:40:06.351672 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.349062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53857 (local address 127.7.105.67:39559)
0809 00:40:06.349111 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.349113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.349198 (+ 85us) server_negotiation.cc:407] Connection header received
0809 00:40:06.349262 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.349263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.349273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.349328 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:06.349490 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.349974 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.350899 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.351277 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.351363 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.351372 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.351560 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.351611 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:40:06.351739 23430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eb9dfb3a494b2d87af2a38851cff1b" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949919c78932479794df00fa2197fb10" is_pre_election: true
I20260809 00:40:06.351900 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.348916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39559 (local address 127.7.105.65:53857)
0809 00:40:06.349113 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.349123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.349204 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.349344 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.349345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.349346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:06.349420 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.349422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.350089 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.350090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.350804 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.350806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.351369 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.351370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.351447 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.351451 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.351776 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.351827 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":132,"threads_started":1}
I20260809 00:40:06.351933 23430 raft_consensus.cc:2454] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 0.
I20260809 00:40:06.352200 23216 leader_election.cc:302] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584645ed78c649f9a51e34588665574b, 949919c78932479794df00fa2197fb10; no voters:
I20260809 00:40:06.352264 23573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eb9dfb3a494b2d87af2a38851cff1b" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e903de616f7475ab45b506cb1b1d94a" is_pre_election: true
I20260809 00:40:06.352353 23647 raft_consensus.cc:2781] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:40:06.352411 23647 raft_consensus.cc:482] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:40:06.352444 23647 raft_consensus.cc:3037] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:06.352438 23573 raft_consensus.cc:2454] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 0.
I20260809 00:40:06.353219 23647 raft_consensus.cc:504] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.353363 23647 leader_election.cc:288] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 election: Requested vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:06.353575 23430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eb9dfb3a494b2d87af2a38851cff1b" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949919c78932479794df00fa2197fb10"
I20260809 00:40:06.353609 23573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eb9dfb3a494b2d87af2a38851cff1b" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e903de616f7475ab45b506cb1b1d94a"
I20260809 00:40:06.353652 23430 raft_consensus.cc:3037] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:06.353688 23573 raft_consensus.cc:3037] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:06.354388 23430 raft_consensus.cc:2454] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 1.
I20260809 00:40:06.354501 23573 raft_consensus.cc:2454] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 1.
I20260809 00:40:06.354547 23216 leader_election.cc:302] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584645ed78c649f9a51e34588665574b, 949919c78932479794df00fa2197fb10; no voters:
I20260809 00:40:06.354647 23647 raft_consensus.cc:2781] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:06.354851 23647 raft_consensus.cc:686] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 LEADER]: Becoming Leader. State: Replica: 584645ed78c649f9a51e34588665574b, State: Running, Role: LEADER
I20260809 00:40:06.354951 23647 consensus_queue.cc:237] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.355796 23121 catalog_manager.cc:5495] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b reported cstate change: term changed from 0 to 1, leader changed from <none> to 584645ed78c649f9a51e34588665574b (127.7.105.65). New cstate: current_term: 1 leader_uuid: "584645ed78c649f9a51e34588665574b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } health_report { overall_health: UNKNOWN } } }
I20260809 00:40:06.361907 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.359033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.7.105.65:35877)
0809 00:40:06.359105 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.359107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.359116 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:40:06.359143 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.359144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.359161 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.359223 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:06.359391 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.359931 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.360768 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.361107 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.361215 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.361457 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:06.361632 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:40:06.361725 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.361748 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.361766 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 00:40:06.363720 23480 tablet.cc:2369] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:40:06.363920 23430 raft_consensus.cc:1261] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Refusing update from remote peer 584645ed78c649f9a51e34588665574b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:06.364259 23573 raft_consensus.cc:1261] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Refusing update from remote peer 584645ed78c649f9a51e34588665574b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:06.364295 23650 consensus_queue.cc:1035] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [LEADER]: Connected to new peer: Peer: permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:06.364522 23650 consensus_queue.cc:1035] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:06.366707 23654 mvcc.cc:205] Tried to move back new op lower bound from 7316422682063482880 to 7316422682030624768. Current Snapshot: MvccSnapshot[applied={T|T < 7316422682063482880}]
I20260809 00:40:06.367365 23656 mvcc.cc:205] Tried to move back new op lower bound from 7316422682063482880 to 7316422682030624768. Current Snapshot: MvccSnapshot[applied={T|T < 7316422682063482880}]
I20260809 00:40:06.369277 23119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:06.369378 23119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:06.370203 23119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Sending DeleteTablet for 3 replicas of tablet 43eb9dfb3a494b2d87af2a38851cff1b
I20260809 00:40:06.370564 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23194
I20260809 00:40:06.370555 23409 tablet_service.cc:1514] Processing DeleteTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:06 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:33928
I20260809 00:40:06.370685 23550 tablet_service.cc:1514] Processing DeleteTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:06 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:41636
I20260809 00:40:06.371697 23666 tablet_replica.cc:330] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: stopping tablet replica
I20260809 00:40:06.371789 23666 raft_consensus.cc:2229] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:06.371881 23666 raft_consensus.cc:2258] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:06.372783 23665 tablet_replica.cc:330] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: stopping tablet replica
I20260809 00:40:06.372885 23665 raft_consensus.cc:2229] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:06.372866 23666 ts_tablet_manager.cc:1905] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:06.372984 23665 raft_consensus.cc:2258] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:06.373361 23665 ts_tablet_manager.cc:1905] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:06.374927 23665 ts_tablet_manager.cc:1918] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:06.374989 23665 log.cc:1196] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/43eb9dfb3a494b2d87af2a38851cff1b
I20260809 00:40:06.375146 23666 ts_tablet_manager.cc:1918] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:06.375254 23666 log.cc:1196] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/43eb9dfb3a494b2d87af2a38851cff1b
I20260809 00:40:06.375337 23665 ts_tablet_manager.cc:1939] T 43eb9dfb3a494b2d87af2a38851cff1b P 949919c78932479794df00fa2197fb10: Deleting consensus metadata
I20260809 00:40:06.375600 23666 ts_tablet_manager.cc:1939] T 43eb9dfb3a494b2d87af2a38851cff1b P 8e903de616f7475ab45b506cb1b1d94a: Deleting consensus metadata
I20260809 00:40:06.375766 23106 catalog_manager.cc:4841] TS 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): tablet 43eb9dfb3a494b2d87af2a38851cff1b (table test-table [id=6e5b163e477d494bab840e782a6ba843]) successfully deleted
I20260809 00:40:06.376217 23106 catalog_manager.cc:4841] TS 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): tablet 43eb9dfb3a494b2d87af2a38851cff1b (table test-table [id=6e5b163e477d494bab840e782a6ba843]) successfully deleted
W20260809 00:40:06.380796 23106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:35877 (error 108)
I20260809 00:40:06.381012 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23341
I20260809 00:40:06.382161 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.381991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35877 (local address 127.0.0.1:48302)
0809 00:40:06.382061 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.382115 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 00:40:06.383503 23106 catalog_manager.cc:4584] TS 584645ed78c649f9a51e34588665574b (127.7.105.65:35877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43eb9dfb3a494b2d87af2a38851cff1b: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35877: connect: Connection refused (error 111)
I20260809 00:40:06.387004 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23484
I20260809 00:40:06.393286 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23087
I20260809 00:40:06.400344 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--webserver_interface=127.7.105.126
--webserver_port=33923
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:06.484831 23668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.485085 23668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.485124 23668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.486291 23668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:06.486361 23668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.486384 23668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.486406 23668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:06.486431 23668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:06.487943 23668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=33923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.488236 23668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.488492 23668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.500041 23677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.500041 23675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.500140 23674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.500329 23668 server_base.cc:1034] running on GCE node
I20260809 00:40:06.500590 23668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.500911 23668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.502087 23668 hybrid_clock.cc:648] HybridClock initialized: now 1786236006502058 us; error 44 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.504698 23668 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:06.505184 23668 webserver.cc:460] Webserver started at http://127.7.105.126:33923/ using document root <none> and password file <none>
I20260809 00:40:06.505404 23668 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.505457 23668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.506827 23668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:40:06.507525 23686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.507718 23668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:06.507830 23668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "86d79253f6654d3fbc1c7ccfac7641c2"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "da727633914708bd1802ea47f0656d7f"
server_key_iv: "481076a40d0b5894fbf454cbf13a7ba8"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.508303 23668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.518592 23668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.520627 23668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.520785 23668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.524981 23668 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44961
I20260809 00:40:06.525017 23742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44961 every 8 connection(s)
I20260809 00:40:06.525436 23668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:06.525744 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23668
I20260809 00:40:06.526046 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:35877
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=40995
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:06.527235 23743 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:40:06.527922 23743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap starting.
I20260809 00:40:06.539927 23743 log.cc:824] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:06.542517 23743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:06.542889 23743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap complete.
I20260809 00:40:06.546345 23743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:06.546674 23743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Initialized, Role: FOLLOWER
I20260809 00:40:06.546836 23743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:06.546901 23743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:06.546952 23743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:06.547010 23743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:40:06.548511 23743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:06.548633 23743 leader_election.cc:302] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86d79253f6654d3fbc1c7ccfac7641c2; no voters:
I20260809 00:40:06.548804 23743 leader_election.cc:288] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:40:06.548949 23747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:40:06.549052 23743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:06.549168 23747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 LEADER]: Becoming Leader. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Running, Role: LEADER
I20260809 00:40:06.549353 23747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:06.549780 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:06.549886 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:06.550344 23757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:06.550624 23749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d79253f6654d3fbc1c7ccfac7641c2. Latest consensus state: current_term: 2 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:06.550719 23749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:06.551365 23757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:06.551496 23757 catalog_manager.cc:1259] Loaded cluster ID: fa8bb02caf094e37ace5de2150d3bc45
I20260809 00:40:06.551537 23757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:40:06.551785 23757 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 00:40:06.557281 23757 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 00:40:06.560367 23757 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:06.560627 23757 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Loaded TSK: 0
I20260809 00:40:06.560877 23757 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:40:06.623864 23745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.624117 23745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.624157 23745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.625303 23745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.625341 23745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.625368 23745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:40:06.626842 23745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:35877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=40995
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.627063 23745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.627324 23745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.632447 23774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.632484 23773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.632472 23776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.633031 23745 server_base.cc:1034] running on GCE node
I20260809 00:40:06.633215 23745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.633476 23745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.634631 23745 hybrid_clock.cc:648] HybridClock initialized: now 1786236006634614 us; error 35 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.637413 23745 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:40:06.637876 23745 webserver.cc:460] Webserver started at http://127.7.105.65:40995/ using document root <none> and password file <none>
I20260809 00:40:06.638139 23745 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.638232 23745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.639683 23745 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:06.640491 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.640681 23745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:06.640789 23745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "584645ed78c649f9a51e34588665574b"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "d48f245ebad1d33579ff45d42ec32c4b"
server_key_iv: "7efc5f7ddf499ea93752b3b5656ecf53"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.641135 23745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.658317 23745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.660737 23745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.660908 23745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.661181 23745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:06.661661 23791 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:40:06.663863 23745 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:40:06.663932 23745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:40:06.663981 23745 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:40:06.664685 23745 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:40:06.664744 23745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:06.664826 23791 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Bootstrap starting.
I20260809 00:40:06.672369 23745 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35877
I20260809 00:40:06.672523 23909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35877 every 8 connection(s)
I20260809 00:40:06.672829 23745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:40:06.677008 23791 log.cc:824] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:06.679800 23791 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:06.680154 23791 tablet_bootstrap.cc:492] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Bootstrap complete.
I20260809 00:40:06.681499 23791 ts_tablet_manager.cc:1397] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.015s
I20260809 00:40:06.683077 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23745
I20260809 00:40:06.682951 23791 raft_consensus.cc:348] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.683341 23791 raft_consensus.cc:729] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584645ed78c649f9a51e34588665574b, State: Initialized, Role: FOLLOWER
I20260809 00:40:06.683373 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:35857
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=39719
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:06.683516 23791 consensus_queue.cc:260] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } }
I20260809 00:40:06.683823 23791 ts_tablet_manager.cc:1428] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:06.688795 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.673555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:55269)
0809 00:40:06.673759 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.673804 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.674290 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.676639 (+ 2349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.676643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.676933 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.677154 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.677161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.679251 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.679252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.680581 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.680585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.681552 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.681553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.681712 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.682355 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.682366 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.684833 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.687103 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.687111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.687113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.687334 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.687626 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.687627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.687628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.687737 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.688167 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.688174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.688311 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.688419 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.688593 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":127,"threads_started":1}
I20260809 00:40:06.688808 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.673684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:55269 (local address 127.7.105.126:44961)
0809 00:40:06.676216 (+ 2532us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.676226 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.676271 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:40:06.676403 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.676409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.676451 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.676586 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.677260 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.679112 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.680680 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.681358 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.681456 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.684968 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.684973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.684973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.684985 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.686996 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.687434 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.687437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.687438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.687537 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.687813 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.687814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.687815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.688106 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.688187 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.688474 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.688604 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2472,"thread_start_us":102,"threads_started":1}
I20260809 00:40:06.689666 23910 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:06.689757 23910 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:06.689932 23910 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:06.690668 23704 ts_manager.cc:194] Registered new tserver with Master: 584645ed78c649f9a51e34588665574b (127.7.105.65:35877)
I20260809 00:40:06.691769 23704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:55269
I20260809 00:40:06.692843 23910 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:40:06.700464 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.691496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.7.105.65:35877)
0809 00:40:06.692373 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.692378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.692386 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:06.692416 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.692417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.692456 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.693082 (+ 626us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.693136 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.693868 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.694766 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.695070 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.695152 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.697290 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.697299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.697299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.697306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.699127 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.699511 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.699514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.699515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.699586 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.699794 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.699795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.699795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.699936 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.700047 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.700266 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.700355 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":64,"threads_started":1}
I20260809 00:40:06.700472 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.691259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35877 (local address 127.0.0.1:48306)
0809 00:40:06.692153 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.692188 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.692247 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.692548 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.692554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.692913 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.692995 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.692998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.693956 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.693957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.694638 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.694641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.695164 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.695165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.695252 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.695750 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.695754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.697199 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.699209 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.699216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.699218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.699415 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.699662 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.699663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.699663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.699723 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.700017 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.700025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.700126 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.700228 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.700355 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":91,"threads_started":1}
I20260809 00:40:06.701242 23836 tablet_service.cc:1514] Processing DeleteTablet for tablet 43eb9dfb3a494b2d87af2a38851cff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:06 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:48306
I20260809 00:40:06.701772 23920 tablet_replica.cc:330] stopping tablet replica
I20260809 00:40:06.701895 23920 raft_consensus.cc:2229] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:06.701972 23920 raft_consensus.cc:2258] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:06.702283 23920 ts_tablet_manager.cc:1905] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:06.704319 23920 ts_tablet_manager.cc:1918] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:06.704406 23920 log.cc:1196] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/43eb9dfb3a494b2d87af2a38851cff1b
I20260809 00:40:06.704643 23920 ts_tablet_manager.cc:1939] T 43eb9dfb3a494b2d87af2a38851cff1b P 584645ed78c649f9a51e34588665574b: Deleting consensus metadata
I20260809 00:40:06.705154 23688 catalog_manager.cc:4841] TS 584645ed78c649f9a51e34588665574b (127.7.105.65:35877): tablet 43eb9dfb3a494b2d87af2a38851cff1b (table test-table [id=6e5b163e477d494bab840e782a6ba843]) successfully deleted
W20260809 00:40:06.791045 23916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.791258 23916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.791281 23916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.792390 23916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.792428 23916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.792454 23916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:40:06.793843 23916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:35857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=39719
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.794049 23916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.794224 23916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.799420 23929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.799397 23925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.799388 23927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.799434 23916 server_base.cc:1034] running on GCE node
I20260809 00:40:06.799804 23916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.800098 23916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.801251 23916 hybrid_clock.cc:648] HybridClock initialized: now 1786236006801246 us; error 37 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.803726 23916 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:40:06.804229 23916 webserver.cc:460] Webserver started at http://127.7.105.66:39719/ using document root <none> and password file <none>
I20260809 00:40:06.804473 23916 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.804557 23916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.805923 23916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:06.806533 23937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.806718 23916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:40:06.806833 23916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "949919c78932479794df00fa2197fb10"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "9d623781f314866c1bc2c165a08c5e78"
server_key_iv: "5ccad6158889590b6ad7ba7dd3fab1d8"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.807238 23916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.822341 23916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.824811 23916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.824986 23916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.825238 23916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:06.825563 23916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:06.825634 23916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.825676 23916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:06.825706 23916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.831588 23916 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35857
I20260809 00:40:06.831660 24065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35857 every 8 connection(s)
I20260809 00:40:06.832041 23916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:06.840591 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 23916
I20260809 00:40:06.840947 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:39559
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=46045
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:06.841796 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.832914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:55485 (local address 127.7.105.126:44961)
0809 00:40:06.832982 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.832984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.833199 (+ 215us) server_negotiation.cc:407] Connection header received
0809 00:40:06.833681 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.833681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.833694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.833751 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.834675 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.835177 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.836207 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.836423 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.836480 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.838786 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.838788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.838788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.838795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.840241 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.840641 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.840642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.840643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.840726 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.841070 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.841071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.841071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.841232 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.841300 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.841650 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.841727 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:40:06.842116 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.832905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:55485)
0809 00:40:06.833061 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.833106 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.833598 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.833811 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.833815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.834380 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.834591 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.834595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.835272 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.835273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.836132 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.836134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.836504 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.836505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.836626 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.837217 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.837226 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.838690 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.840313 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.840320 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.840322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.840546 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.840811 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.840812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.840812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.840988 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.841312 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.841320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.841497 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.841629 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.841827 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260809 00:40:06.842724 24066 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:06.842844 24066 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:06.843073 24066 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:06.843600 23704 ts_manager.cc:194] Registered new tserver with Master: 949919c78932479794df00fa2197fb10 (127.7.105.66:35857)
I20260809 00:40:06.844163 23704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:55485
W20260809 00:40:06.927570 24069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:06.927783 24069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:06.927806 24069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:06.929008 24069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:06.929050 24069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:06.929077 24069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:40:06.930460 24069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:39559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=46045
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:06.930639 24069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:06.930855 24069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:06.936019 24076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.936152 24078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:06.936074 24075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:06.936216 24069 server_base.cc:1034] running on GCE node
I20260809 00:40:06.936482 24069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:06.936767 24069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:06.937929 24069 hybrid_clock.cc:648] HybridClock initialized: now 1786236006937910 us; error 39 us; skew 500 ppm
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:06.940454 24069 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:40:06.940879 24069 webserver.cc:460] Webserver started at http://127.7.105.67:46045/ using document root <none> and password file <none>
I20260809 00:40:06.941071 24069 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:06.941124 24069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:06.942438 24069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:06.942991 24085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.943157 24069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.943251 24069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "8e903de616f7475ab45b506cb1b1d94a"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "4596cbbd111c1db106b2bc496a2132b9"
server_key_iv: "3c78958f34fb564d28540eddc4525ef3"
server_key_version: "encryptionkey@0"
I20260809 00:40:06.943593 24069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:06.975695 24069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:06.978195 24069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:06.978389 24069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:06.978667 24069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:06.978994 24069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:06.979028 24069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.979053 24069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:06.979097 24069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:06.984905 24069 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:39559
I20260809 00:40:06.985069 24209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:39559 every 8 connection(s)
I20260809 00:40:06.985343 24069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:40:06.988163 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24069
Aug 09 00:40:06 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:06.996759 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.988751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.7.105.126:44961)
0809 00:40:06.988879 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.988881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.988891 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:40:06.989043 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.989043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.989053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.989103 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.989884 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.990343 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.991492 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.991689 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.991740 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.993123 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.993125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.993125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.993134 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.995124 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.995528 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.995530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.995530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.995625 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.996000 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.996001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.996002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.996213 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.996439 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.996481 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.996587 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260809 00:40:06.996771 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.986145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:42355 (local address 127.7.105.126:44961)
0809 00:40:06.986196 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:06.986198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:06.986435 (+ 237us) server_negotiation.cc:407] Connection header received
0809 00:40:06.987020 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:06.987021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:06.987034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:06.987090 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:06.988179 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.988911 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.990479 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.990659 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.990711 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.993264 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:06.993265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:06.993265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:06.993273 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:06.995164 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.995669 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.995670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.995670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.995729 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:06.995987 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:06.995988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:06.995989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:06.996173 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:06.996243 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:06.996525 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 00:40:06.996656 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":21}
I20260809 00:40:06.996932 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:06.986107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:42355)
0809 00:40:06.986307 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:06.986345 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:06.986905 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:06.987160 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:06.987164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:06.987770 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:06.988034 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.988041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.989022 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.989023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.990349 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:06.990352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:06.990755 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:06.990756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:06.990929 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:06.991485 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:06.991497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:06.993166 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:06.995290 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.995297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.995299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.995523 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.995799 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:06.995800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:06.995801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:06.995910 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:06.996248 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:06.996255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:06.996392 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:40:06.996503 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:40:06.996673 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260809 00:40:06.997424 24210 heartbeater.cc:347] Connected to a master server at 127.7.105.126:44961
I20260809 00:40:06.997565 24210 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:06.997802 24210 heartbeater.cc:511] Master 127.7.105.126:44961 requested a full tablet report, sending...
I20260809 00:40:06.998178 23704 ts_manager.cc:194] Registered new tserver with Master: 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:06.998553 23704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:42355
I20260809 00:40:07.009207 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:40:07.012704 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.009648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.7.105.126:44961)
0809 00:40:07.009733 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.009735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.009836 (+ 101us) server_negotiation.cc:407] Connection header received
0809 00:40:07.009957 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.009958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.010026 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.010120 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:07.010289 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.010871 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.011770 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.012103 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.012168 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.012343 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:07.012464 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:40:07.012564 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.012589 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.012616 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:40:07.013027 23704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:40:07.014674 23704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:40:07.017407 23836 tablet_service.cc:1467] Processing CreateTablet for tablet 27642c39bc81494aa79cea160acb0773 (DEFAULT_TABLE table=test-table [id=b8945821c7fc45698254b523473836a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:07.017642 23836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27642c39bc81494aa79cea160acb0773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:07.019253 23791 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Bootstrap starting.
I20260809 00:40:07.019616 23791 tablet_bootstrap.cc:654] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:07.020524 23791 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: No bootstrap required, opened a new log
I20260809 00:40:07.020591 23791 ts_tablet_manager.cc:1397] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:07.020759 23791 raft_consensus.cc:348] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.020834 23791 raft_consensus.cc:374] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:07.020879 23791 raft_consensus.cc:729] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584645ed78c649f9a51e34588665574b, State: Initialized, Role: FOLLOWER
I20260809 00:40:07.020928 23791 consensus_queue.cc:260] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.021011 23791 ts_tablet_manager.cc:1428] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:40:07.022368 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.016979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39559 (local address 127.0.0.1:41640)
0809 00:40:07.017092 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.017102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:07.017149 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:07.017761 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:07.017762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:07.017765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:07.017812 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.017814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.019138 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.019139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.020628 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.020631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.022117 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.022118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.022179 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.022205 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 00:40:07.022254 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:40:07.022297 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 00:40:07.022406 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.017275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.7.105.67:39559)
0809 00:40:07.017486 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.017492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.017504 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:07.017546 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.017548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.017564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.017663 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:07.017899 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.019032 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.020742 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.021514 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.021594 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.021655 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.022285 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.022328 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260809 00:40:07.023123 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.017038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.7.105.66:35857)
0809 00:40:07.019342 (+ 2304us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.019347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.019358 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:40:07.019405 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.019407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.019424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.019514 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:07.019665 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.020814 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.022133 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.022798 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.022859 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.022897 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.023032 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.023061 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2263,"thread_start_us":110,"threads_started":1}
I20260809 00:40:07.023149 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.016880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35857 (local address 127.0.0.1:33932)
0809 00:40:07.016944 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.016957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:07.017002 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:07.019517 (+ 2515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:07.019518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:07.019520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:07.019554 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.019555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.020980 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.020981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.022029 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.022031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.022881 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.022882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.022952 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.022956 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:07.023022 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:40:07.023074 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 00:40:07.023134 24137 tablet_service.cc:1467] Processing CreateTablet for tablet 27642c39bc81494aa79cea160acb0773 (DEFAULT_TABLE table=test-table [id=b8945821c7fc45698254b523473836a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:07.023547 24137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27642c39bc81494aa79cea160acb0773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:07.023857 23991 tablet_service.cc:1467] Processing CreateTablet for tablet 27642c39bc81494aa79cea160acb0773 (DEFAULT_TABLE table=test-table [id=b8945821c7fc45698254b523473836a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:07.024219 23991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27642c39bc81494aa79cea160acb0773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:07.027201 24220 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Bootstrap starting.
I20260809 00:40:07.027725 24221 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Bootstrap starting.
I20260809 00:40:07.027865 24220 tablet_bootstrap.cc:654] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:07.028509 24221 tablet_bootstrap.cc:654] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:07.028528 24220 log.cc:824] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:07.029060 24221 log.cc:824] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:07.029340 24220 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: No bootstrap required, opened a new log
I20260809 00:40:07.029417 24220 ts_tablet_manager.cc:1397] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:07.029836 24221 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: No bootstrap required, opened a new log
I20260809 00:40:07.029908 24221 ts_tablet_manager.cc:1397] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:40:07.031388 24220 raft_consensus.cc:348] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.031585 24220 raft_consensus.cc:374] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:07.031626 24220 raft_consensus.cc:729] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e903de616f7475ab45b506cb1b1d94a, State: Initialized, Role: FOLLOWER
I20260809 00:40:07.031771 24220 consensus_queue.cc:260] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.031827 24221 raft_consensus.cc:348] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.032007 24221 raft_consensus.cc:374] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:07.032091 24220 ts_tablet_manager.cc:1428] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:40:07.032086 24221 raft_consensus.cc:729] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949919c78932479794df00fa2197fb10, State: Initialized, Role: FOLLOWER
I20260809 00:40:07.032122 24210 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
I20260809 00:40:07.032212 24221 consensus_queue.cc:260] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.032486 24221 ts_tablet_manager.cc:1428] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:40:07.032612 24066 heartbeater.cc:503] Master 127.7.105.126:44961 was elected leader, sending a full tablet report...
W20260809 00:40:07.082762 24067 tablet.cc:2369] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:40:07.176733 23911 tablet.cc:2369] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:40:07.235955 24211 tablet.cc:2369] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:40:07.273903 23915 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:07.274051 23915 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.274483 23915 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:07.277668 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.274847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:54355 (local address 127.7.105.66:35857)
0809 00:40:07.274931 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.274933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.274945 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:07.275149 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.275151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.275161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.275243 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:07.275451 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.275895 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.276904 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.277288 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.277346 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.277376 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.277553 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.277593 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:40:07.277689 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.274992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:43579 (local address 127.7.105.67:39559)
0809 00:40:07.275068 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.275070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.275079 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:40:07.275158 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.275159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.275167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.275211 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:07.275384 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.275883 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.276940 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.277296 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.277386 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.277394 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.277582 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.277622 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:40:07.277693 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.274705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35857 (local address 127.7.105.65:54355)
0809 00:40:07.274864 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.274880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:07.274973 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:07.275267 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:07.275268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:07.275271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:07.275368 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.275371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.276008 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.276009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.276805 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.276808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.277377 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.277378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.277458 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.277477 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 00:40:07.277523 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:40:07.277590 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260809 00:40:07.277696 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.274895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39559 (local address 127.7.105.65:43579)
0809 00:40:07.275000 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.275011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:07.275063 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:07.275234 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:07.275235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:07.275237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:07.275301 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.275303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.275966 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.275968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.276815 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:07.276817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.277410 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.277411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:07.277485 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.277489 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:07.277560 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:40:07.277607 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":22,"thread_start_us":40,"threads_started":1}
I20260809 00:40:07.278275 24160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27642c39bc81494aa79cea160acb0773" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e903de616f7475ab45b506cb1b1d94a" is_pre_election: true
I20260809 00:40:07.278290 24013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27642c39bc81494aa79cea160acb0773" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949919c78932479794df00fa2197fb10" is_pre_election: true
I20260809 00:40:07.278399 24160 raft_consensus.cc:2454] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 0.
I20260809 00:40:07.278445 24013 raft_consensus.cc:2454] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 0.
I20260809 00:40:07.278646 23786 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584645ed78c649f9a51e34588665574b, 8e903de616f7475ab45b506cb1b1d94a; no voters:
I20260809 00:40:07.278817 23915 raft_consensus.cc:2781] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:40:07.278882 23915 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:40:07.278898 23915 raft_consensus.cc:3037] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:07.279570 23915 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.279677 23915 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 election: Requested vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
I20260809 00:40:07.279937 24160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27642c39bc81494aa79cea160acb0773" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e903de616f7475ab45b506cb1b1d94a"
I20260809 00:40:07.280012 24160 raft_consensus.cc:3037] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:07.279979 24013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27642c39bc81494aa79cea160acb0773" candidate_uuid: "584645ed78c649f9a51e34588665574b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949919c78932479794df00fa2197fb10"
I20260809 00:40:07.280102 24013 raft_consensus.cc:3037] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:07.280764 24160 raft_consensus.cc:2454] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 1.
I20260809 00:40:07.280961 24013 raft_consensus.cc:2454] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584645ed78c649f9a51e34588665574b in term 1.
I20260809 00:40:07.280992 23786 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584645ed78c649f9a51e34588665574b, 8e903de616f7475ab45b506cb1b1d94a; no voters:
I20260809 00:40:07.281076 23915 raft_consensus.cc:2781] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:07.281200 23915 raft_consensus.cc:686] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 LEADER]: Becoming Leader. State: Replica: 584645ed78c649f9a51e34588665574b, State: Running, Role: LEADER
I20260809 00:40:07.281272 23915 consensus_queue.cc:237] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.282243 23703 catalog_manager.cc:5495] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b reported cstate change: term changed from 0 to 1, leader changed from <none> to 584645ed78c649f9a51e34588665574b (127.7.105.65). New cstate: current_term: 1 leader_uuid: "584645ed78c649f9a51e34588665574b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } health_report { overall_health: HEALTHY } } }
I20260809 00:40:07.304049 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.301573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.7.105.65:35877)
0809 00:40:07.301765 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:07.301767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:07.301775 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:07.301815 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:07.301816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:07.301836 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:07.301927 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:07.302045 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.302522 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.303246 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:07.303438 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:07.303494 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:07.303703 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:07.303802 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:40:07.303856 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:07.303925 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:40:07.303943 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":123,"threads_started":1}
I20260809 00:40:07.305702 24160 raft_consensus.cc:1261] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Refusing update from remote peer 584645ed78c649f9a51e34588665574b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:07.305778 24013 raft_consensus.cc:1261] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Refusing update from remote peer 584645ed78c649f9a51e34588665574b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:07.306092 24231 consensus_queue.cc:1035] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:07.306277 23915 consensus_queue.cc:1035] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [LEADER]: Connected to new peer: Peer: permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:07.309324 24237 mvcc.cc:205] Tried to move back new op lower bound from 7316422685921521664 to 7316422685825118208. Current Snapshot: MvccSnapshot[applied={T|T < 7316422685921521664}]
I20260809 00:40:07.310016 24236 mvcc.cc:205] Tried to move back new op lower bound from 7316422685921521664 to 7316422685825118208. Current Snapshot: MvccSnapshot[applied={T|T < 7316422685921521664}]
I20260809 00:40:07.311951 23704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:07.312052 23704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:07.312850 23704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Sending DeleteTablet for 3 replicas of tablet 27642c39bc81494aa79cea160acb0773
I20260809 00:40:07.313184 23991 tablet_service.cc:1514] Processing DeleteTablet for tablet 27642c39bc81494aa79cea160acb0773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:33932
I20260809 00:40:07.313227 24137 tablet_service.cc:1514] Processing DeleteTablet for tablet 27642c39bc81494aa79cea160acb0773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260809 00:40:07.313242 23836 tablet_service.cc:1514] Processing DeleteTablet for tablet 27642c39bc81494aa79cea160acb0773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:07 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:48306
I20260809 00:40:07.313608 24248 tablet_replica.cc:330] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: stopping tablet replica
I20260809 00:40:07.313650 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23745
I20260809 00:40:07.313673 24248 raft_consensus.cc:2229] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:40:07.313755 24248 raft_consensus.cc:2258] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:07.315117 24245 tablet_replica.cc:330] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: stopping tablet replica
I20260809 00:40:07.315117 24247 tablet_replica.cc:330] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: stopping tablet replica
I20260809 00:40:07.315217 24247 raft_consensus.cc:2229] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:07.315217 24245 raft_consensus.cc:2229] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:07.315315 24247 raft_consensus.cc:2258] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:07.315315 24245 raft_consensus.cc:2258] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:07.315855 24247 ts_tablet_manager.cc:1905] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:07.316215 24245 ts_tablet_manager.cc:1905] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:07.317646 24247 ts_tablet_manager.cc:1918] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:07.317730 24247 log.cc:1196] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/27642c39bc81494aa79cea160acb0773
I20260809 00:40:07.317869 24245 ts_tablet_manager.cc:1918] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:07.318044 24247 ts_tablet_manager.cc:1939] T 27642c39bc81494aa79cea160acb0773 P 8e903de616f7475ab45b506cb1b1d94a: Deleting consensus metadata
I20260809 00:40:07.318212 24245 log.cc:1196] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/27642c39bc81494aa79cea160acb0773
I20260809 00:40:07.318460 23688 catalog_manager.cc:4841] TS 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): tablet 27642c39bc81494aa79cea160acb0773 (table test-table [id=b8945821c7fc45698254b523473836a2]) successfully deleted
I20260809 00:40:07.318485 24245 ts_tablet_manager.cc:1939] T 27642c39bc81494aa79cea160acb0773 P 949919c78932479794df00fa2197fb10: Deleting consensus metadata
I20260809 00:40:07.318917 23688 catalog_manager.cc:4841] TS 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): tablet 27642c39bc81494aa79cea160acb0773 (table test-table [id=b8945821c7fc45698254b523473836a2]) successfully deleted
W20260809 00:40:07.324003 23688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:35877 (error 108)
I20260809 00:40:07.324355 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23916
I20260809 00:40:07.324553 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.324407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:35877 (local address 127.0.0.1:48322)
0809 00:40:07.324474 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.324514 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 00:40:07.324932 23688 catalog_manager.cc:4584] TS 584645ed78c649f9a51e34588665574b (127.7.105.65:35877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27642c39bc81494aa79cea160acb0773: Network error: Client connection negotiation failed: client connection to 127.7.105.65:35877: connect: Connection refused (error 111)
I20260809 00:40:07.333429 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24069
I20260809 00:40:07.339669 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 23668
I20260809 00:40:07.346251 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--webserver_interface=127.7.105.126
--webserver_port=33923
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:07.430477 24250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:07.430706 24250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:07.430729 24250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:07.431926 24250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:07.431972 24250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:07.431986 24250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:07.431999 24250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:07.432011 24250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:07.433652 24250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=33923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:07.433920 24250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:07.434175 24250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:07.439359 24255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.439424 24256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.439368 24259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:07.439698 24250 server_base.cc:1034] running on GCE node
I20260809 00:40:07.440011 24250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:07.440351 24250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:07.441519 24250 hybrid_clock.cc:648] HybridClock initialized: now 1786236007441508 us; error 45 us; skew 500 ppm
Aug 09 00:40:07 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:07.444135 24250 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:07.444618 24250 webserver.cc:460] Webserver started at http://127.7.105.126:33923/ using document root <none> and password file <none>
I20260809 00:40:07.444834 24250 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:07.444877 24250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:07.446187 24250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:07.446833 24267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.447063 24250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:07.447166 24250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "86d79253f6654d3fbc1c7ccfac7641c2"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "da727633914708bd1802ea47f0656d7f"
server_key_iv: "481076a40d0b5894fbf454cbf13a7ba8"
server_key_version: "encryptionkey@0"
I20260809 00:40:07.447518 24250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:07.472450 24250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:07.474516 24250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:07.474676 24250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:07.479039 24250 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44961
I20260809 00:40:07.479091 24325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44961 every 8 connection(s)
I20260809 00:40:07.479508 24250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:07.481169 24327 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:40:07.481942 24327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap starting.
I20260809 00:40:07.482020 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24250
I20260809 00:40:07.482326 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:35877
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=40995
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:07.492141 24327 log.cc:824] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:07.494481 24327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:07.494815 24327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: Bootstrap complete.
I20260809 00:40:07.497385 24327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:07.497691 24327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Initialized, Role: FOLLOWER
I20260809 00:40:07.497906 24327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:07.498041 24327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:07.498108 24327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:07.498162 24327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:40:07.499370 24327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:07.499490 24327 leader_election.cc:302] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86d79253f6654d3fbc1c7ccfac7641c2; no voters:
I20260809 00:40:07.499701 24327 leader_election.cc:288] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:40:07.499737 24331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:40:07.499845 24331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [term 3 LEADER]: Becoming Leader. State: Replica: 86d79253f6654d3fbc1c7ccfac7641c2, State: Running, Role: LEADER
I20260809 00:40:07.499943 24331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } }
I20260809 00:40:07.499980 24327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:07.500253 24332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:07.500355 24332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:07.500248 24331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d79253f6654d3fbc1c7ccfac7641c2. Latest consensus state: current_term: 3 leader_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d79253f6654d3fbc1c7ccfac7641c2" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44961 } } }
I20260809 00:40:07.500566 24331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2 [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:07.501881 24349 catalog_manager.cc:1259] Loaded cluster ID: fa8bb02caf094e37ace5de2150d3bc45
I20260809 00:40:07.501932 24349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: loading cluster ID for follower catalog manager: success
W20260809 00:40:07.502244 24349 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 86d79253f6654d3fbc1c7ccfac7641c2: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260809 00:40:07.502292 24349 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260809 00:40:07.502326 24336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:07.502722 24336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:07.502806 24336 catalog_manager.cc:1259] Loaded cluster ID: fa8bb02caf094e37ace5de2150d3bc45
I20260809 00:40:07.502841 24336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 00:40:07.502974 24336 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9856be59ad google::LogMessage::Fail() at ??:0
@ 0x7f9856be888a google::LogMessage::SendToLog() at ??:0
@ 0x7f9856be54de google::LogMessage::Flush() at ??:0
@ 0x7f9856be6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f985eb060f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f985eb0f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9858ab30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9858aa8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f985990c6db start_thread at ??:0
@ 0x7f98556e361f clone at ??:0
W20260809 00:40:07.579452 24329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:07.579668 24329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:07.579692 24329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:07.580914 24329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:07.580958 24329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:07.580986 24329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:40:07.582460 24329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:35877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=40995
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:07.582719 24329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:07.582957 24329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:07.588373 24367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.588436 24366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.588382 24369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:07.588609 24329 server_base.cc:1034] running on GCE node
I20260809 00:40:07.588831 24329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:07.589113 24329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:07.590306 24329 hybrid_clock.cc:648] HybridClock initialized: now 1786236007590282 us; error 50 us; skew 500 ppm
Aug 09 00:40:07 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:07.593554 24329 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:40:07.594172 24329 webserver.cc:460] Webserver started at http://127.7.105.65:40995/ using document root <none> and password file <none>
I20260809 00:40:07.594465 24329 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:07.594529 24329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:07.596396 24329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:40:07.597200 24376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.597393 24329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:07.597540 24329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "584645ed78c649f9a51e34588665574b"
format_stamp: "Formatted at 2026-08-09 00:40:05 on dist-test-slave-1zqn"
server_key: "d48f245ebad1d33579ff45d42ec32c4b"
server_key_iv: "7efc5f7ddf499ea93752b3b5656ecf53"
server_key_version: "encryptionkey@0"
I20260809 00:40:07.597900 24329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:07.606350 24329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:07.609135 24329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:07.609304 24329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:07.609578 24329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:07.610105 24385 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:40:07.612152 24329 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:40:07.612228 24329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:40:07.612287 24329 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:40:07.613195 24329 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:40:07.613260 24329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:07.613329 24385 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Bootstrap starting.
I20260809 00:40:07.621374 24329 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:35877
I20260809 00:40:07.621837 24329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:40:07.624307 24500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:35877 every 8 connection(s)
I20260809 00:40:07.624758 24385 log.cc:824] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:07.627588 24385 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:07.627908 24385 tablet_bootstrap.cc:492] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Bootstrap complete.
I20260809 00:40:07.628933 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24329
I20260809 00:40:07.629216 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:35857
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=39719
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:07.629249 24385 ts_tablet_manager.cc:1397] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.014s
I20260809 00:40:07.630822 24385 raft_consensus.cc:348] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.631204 24385 raft_consensus.cc:729] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584645ed78c649f9a51e34588665574b, State: Initialized, Role: FOLLOWER
I20260809 00:40:07.631366 24385 consensus_queue.cc:260] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:07.631666 24385 ts_tablet_manager.cc:1428] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260809 00:40:07.708971 24504 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:40:07.622705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:35251)
0809 00:40:07.622913 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.622961 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:07.623450 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:07.708807 (+ 85357us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
W20260809 00:40:07.709275 24501 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:44961 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:44961: Client connection negotiation failed: client connection to 127.7.105.126:44961: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:40:07.709650 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.709529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:47379)
0809 00:40:07.709584 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.709631 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:40:07.710093 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.709994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:35903)
0809 00:40:07.710042 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.710066 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:40:07.710239 24501 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:40:07.727859 24506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:07.728139 24506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:07.728188 24506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:07.729364 24506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:07.729444 24506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:07.729493 24506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:40:07.730991 24506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:35857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=39719
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:07.731272 24506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:07.731534 24506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:07.736742 24515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.736742 24512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.736775 24513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:07.737104 24506 server_base.cc:1034] running on GCE node
I20260809 00:40:07.737325 24506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:07.737566 24506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:07.738734 24506 hybrid_clock.cc:648] HybridClock initialized: now 1786236007738722 us; error 31 us; skew 500 ppm
Aug 09 00:40:07 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:07.741446 24506 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:40:07.741884 24506 webserver.cc:460] Webserver started at http://127.7.105.66:39719/ using document root <none> and password file <none>
I20260809 00:40:07.742134 24506 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:07.742206 24506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:07.743546 24506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:07.744228 24523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.744412 24506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:07.744514 24506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "949919c78932479794df00fa2197fb10"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "9d623781f314866c1bc2c165a08c5e78"
server_key_iv: "5ccad6158889590b6ad7ba7dd3fab1d8"
server_key_version: "encryptionkey@0"
I20260809 00:40:07.744886 24506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:07.771904 24506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:07.774564 24506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:07.774750 24506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:07.775027 24506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:07.775374 24506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:07.775444 24506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.775482 24506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:07.775530 24506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.781555 24506 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:35857
I20260809 00:40:07.781683 24645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:35857 every 8 connection(s)
I20260809 00:40:07.782017 24506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:40:07.783169 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.782744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:33415)
0809 00:40:07.782909 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.783001 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
W20260809 00:40:07.783576 24646 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:44961 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:44961: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
I20260809 00:40:07.783869 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.783766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:47415)
0809 00:40:07.783821 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.783846 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 00:40:07.784188 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.784118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:55547)
0809 00:40:07.784147 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.784169 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 00:40:07.784299 24646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:40:07.786592 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24506
I20260809 00:40:07.786864 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:39559
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:44961
--webserver_port=46045
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:07.878959 24651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:07.879240 24651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:07.879285 24651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:07.880661 24651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:07.880736 24651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:07.880786 24651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:40:07.882596 24651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:39559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=46045
--tserver_master_addrs=127.7.105.126:44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:07.882915 24651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:07.883193 24651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:07.888774 24659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.888774 24656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:07.888836 24657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:07.889305 24651 server_base.cc:1034] running on GCE node
I20260809 00:40:07.889513 24651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:07.889729 24651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:07.890897 24651 hybrid_clock.cc:648] HybridClock initialized: now 1786236007890891 us; error 37 us; skew 500 ppm
Aug 09 00:40:07 dist-test-slave-1zqn krb5kdc[23072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:07.893590 24651 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:40:07.894093 24651 webserver.cc:460] Webserver started at http://127.7.105.67:46045/ using document root <none> and password file <none>
I20260809 00:40:07.894343 24651 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:07.894416 24651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:07.895781 24651 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:40:07.896459 24667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.896634 24651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.896744 24651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "8e903de616f7475ab45b506cb1b1d94a"
format_stamp: "Formatted at 2026-08-09 00:40:06 on dist-test-slave-1zqn"
server_key: "4596cbbd111c1db106b2bc496a2132b9"
server_key_iv: "3c78958f34fb564d28540eddc4525ef3"
server_key_version: "encryptionkey@0"
I20260809 00:40:07.897102 24651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:07.907764 24651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:07.910118 24651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:07.910285 24651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:07.910517 24651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:07.910836 24651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:07.910892 24651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.910935 24651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:07.910962 24651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:07.917354 24651 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:39559
I20260809 00:40:07.917395 24790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:39559 every 8 connection(s)
I20260809 00:40:07.917768 24651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:40:07.918882 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.918472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:58611)
0809 00:40:07.918657 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.918729 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
W20260809 00:40:07.919443 24791 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:44961 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:44961: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
I20260809 00:40:07.919791 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.919676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:37399)
0809 00:40:07.919714 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.919763 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:40:07.920205 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:07.920124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:36461)
0809 00:40:07.920154 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:07.920188 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 00:40:07.920342 24791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:40:07.922987 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24651
I20260809 00:40:08.710955 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:08.710669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:43423)
0809 00:40:08.710830 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:08.710877 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260809 00:40:08.784910 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:08.784658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:52409)
0809 00:40:08.784811 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:08.784854 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260809 00:40:08.920871 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:08.920639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:49301)
0809 00:40:08.920776 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:08.920816 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260809 00:40:09.148828 24798 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:09.149155 24798 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:09.149560 24798 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
Aug 09 00:40:09 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.67@KRBTEST.COM
Aug 09 00:40:09 dist-test-slave-1zqn krb5kdc[23072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.66@KRBTEST.COM
I20260809 00:40:09.160454 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.149969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:40263 (local address 127.7.105.67:39559)
0809 00:40:09.150143 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:09.150154 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:09.150213 (+ 59us) server_negotiation.cc:407] Connection header received
0809 00:40:09.150701 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:09.150703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:09.150740 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:09.150897 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:09.151680 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.152589 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.153569 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.153983 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.154109 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:09.156547 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:09.156554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:09.156555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:09.156568 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:09.158656 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159130 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:09.159133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:09.159133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:09.159225 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159508 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:09.159509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:09.159509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:09.159766 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:09.159868 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:09.160200 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 00:40:09.160329 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
I20260809 00:40:09.160440 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.149855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:39559 (local address 127.7.105.65:40263)
0809 00:40:09.150145 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:09.150156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:09.150227 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:09.150894 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:09.150895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:09.151334 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:09.151531 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:09.151537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.152708 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.152709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:09.153473 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:09.153476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.154076 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.154077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:09.154208 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:09.154814 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:09.154824 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:09.156424 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:09.158804 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:09.158811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:09.158813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:09.159048 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:09.159316 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:09.159318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:09.159430 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:09.159848 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:09.159854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:09.159988 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 00:40:09.160139 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 00:40:09.160329 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260809 00:40:09.160569 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.149848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:59773 (local address 127.7.105.66:35857)
0809 00:40:09.150023 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:09.150034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:09.150070 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:40:09.150655 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:09.150657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:09.150702 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:09.150815 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:09.151650 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.152578 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.153599 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.153992 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.154126 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:09.156846 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:09.156850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:09.156850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:09.156862 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:09.158666 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159356 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:09.159359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:09.159360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:09.159456 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159676 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:09.159678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:09.159678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:09.159910 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:09.159998 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:09.160260 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 00:40:09.160415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 00:40:09.160969 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.149737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:35857 (local address 127.7.105.65:59773)
0809 00:40:09.149801 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:09.150013 (+ 212us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:09.150084 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:09.150887 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:09.150890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:09.151335 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:09.151531 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:09.151537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.152692 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.152693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:09.153476 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:09.153479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:09.154101 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:09.154102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:09.154209 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:09.154880 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:09.154885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:09.156731 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:09.159127 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:09.159127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:09.159256 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:09.159539 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:09.159540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:09.159540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:09.159607 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:09.159993 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:09.159994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:09.160110 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 00:40:09.160184 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:40:09.160417 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":204}
W20260809 00:40:09.161677 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
W20260809 00:40:09.161819 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
I20260809 00:40:09.161861 24378 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 584645ed78c649f9a51e34588665574b; no voters: 8e903de616f7475ab45b506cb1b1d94a, 949919c78932479794df00fa2197fb10
I20260809 00:40:09.162034 24798 raft_consensus.cc:2726] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:40:09.711642 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.711431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:33551)
0809 00:40:09.711555 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:09.711593 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260809 00:40:09.785677 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.785447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:36055)
0809 00:40:09.785600 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:09.785638 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260809 00:40:09.921550 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:09.921340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:48719)
0809 00:40:09.921470 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:09.921508 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260809 00:40:10.712437 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:10.712213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:48995)
0809 00:40:10.712344 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:10.712387 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260809 00:40:10.786679 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:10.786431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:53269)
0809 00:40:10.786589 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:10.786635 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260809 00:40:10.922271 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:10.922021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:48525)
0809 00:40:10.922185 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:10.922226 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260809 00:40:11.018608 24809 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:11.018725 24809 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:11.018944 24809 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
W20260809 00:40:11.019443 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
W20260809 00:40:11.019542 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
I20260809 00:40:11.019644 24378 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 584645ed78c649f9a51e34588665574b; no voters: 8e903de616f7475ab45b506cb1b1d94a, 949919c78932479794df00fa2197fb10
I20260809 00:40:11.019760 24809 raft_consensus.cc:2726] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:40:11.713199 24810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:11.712944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:42263)
0809 00:40:11.713105 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:11.713148 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260809 00:40:11.787431 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:11.787219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:58603)
0809 00:40:11.787347 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:11.787387 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260809 00:40:11.922927 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:11.922714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:48175)
0809 00:40:11.922844 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:11.922880 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260809 00:40:12.714080 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:12.713797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:60551)
0809 00:40:12.713982 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:12.714031 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260809 00:40:12.788401 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:12.788082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:41505)
0809 00:40:12.788268 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:12.788335 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
I20260809 00:40:12.923879 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:12.923586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:33765)
0809 00:40:12.923768 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:12.923831 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
I20260809 00:40:13.714905 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:13.714648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:45037)
0809 00:40:13.714807 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:13.714853 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260809 00:40:13.789211 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:13.788994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:41837)
0809 00:40:13.789125 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:13.789165 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260809 00:40:13.924674 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:13.924425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:40489)
0809 00:40:13.924585 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:13.924624 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260809 00:40:13.980862 24819 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:13.980981 24819 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:13.981118 24819 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
W20260809 00:40:13.981597 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
W20260809 00:40:13.981681 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
I20260809 00:40:13.981720 24378 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 584645ed78c649f9a51e34588665574b; no voters: 8e903de616f7475ab45b506cb1b1d94a, 949919c78932479794df00fa2197fb10
I20260809 00:40:13.981820 24819 raft_consensus.cc:2726] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:40:14.715653 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:14.715415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:44345)
0809 00:40:14.715559 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:14.715603 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260809 00:40:14.789956 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:14.789716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:40591)
0809 00:40:14.789874 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:14.789912 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260809 00:40:14.925343 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:14.925125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:46327)
0809 00:40:14.925259 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:14.925296 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260809 00:40:15.650372 24823 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:15.650470 24823 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:15.650653 24823 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
W20260809 00:40:15.651038 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
W20260809 00:40:15.651129 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
I20260809 00:40:15.651149 24378 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 584645ed78c649f9a51e34588665574b; no voters: 8e903de616f7475ab45b506cb1b1d94a, 949919c78932479794df00fa2197fb10
I20260809 00:40:15.651208 24823 raft_consensus.cc:2726] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:40:15.716550 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:15.716309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:37261)
0809 00:40:15.716458 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:15.716501 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 00:40:15.790693 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:15.790447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:55923)
0809 00:40:15.790604 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:15.790646 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260809 00:40:15.926072 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:15.925821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:35769)
0809 00:40:15.925981 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:15.926026 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260809 00:40:16.717412 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:16.717022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:52979)
0809 00:40:16.717184 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:16.717349 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260809 00:40:16.791365 24829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:16.791130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:39289)
0809 00:40:16.791264 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:16.791307 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260809 00:40:16.926926 24830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:16.926671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:57857)
0809 00:40:16.926825 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:16.926859 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260809 00:40:17.718417 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:17.718106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:42413)
0809 00:40:17.718260 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:17.718315 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 00:40:17.792140 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:17.791844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:43811)
0809 00:40:17.791998 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:17.792042 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260809 00:40:17.927717 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:17.927491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:58811)
0809 00:40:17.927628 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:17.927669 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 00:40:18.719134 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:18.718893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:45903)
0809 00:40:18.719042 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:18.719085 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260809 00:40:18.792935 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:18.792676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:51941)
0809 00:40:18.792836 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:18.792885 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260809 00:40:18.928476 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:18.928244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:32999)
0809 00:40:18.928390 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:18.928429 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260809 00:40:19.554057 24837 raft_consensus.cc:482] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:19.554198 24837 raft_consensus.cc:504] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949919c78932479794df00fa2197fb10" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 35857 } } peers { permanent_uuid: "8e903de616f7475ab45b506cb1b1d94a" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 39559 } } peers { permanent_uuid: "584645ed78c649f9a51e34588665574b" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 35877 } }
I20260809 00:40:19.554383 24837 leader_election.cc:288] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 949919c78932479794df00fa2197fb10 (127.7.105.66:35857), 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559)
W20260809 00:40:19.554796 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 949919c78932479794df00fa2197fb10 (127.7.105.66:35857): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
W20260809 00:40:19.554880 24378 leader_election.cc:341] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e903de616f7475ab45b506cb1b1d94a (127.7.105.67:39559): Not found: Tablet not found: 27642c39bc81494aa79cea160acb0773
I20260809 00:40:19.554898 24378 leader_election.cc:302] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 584645ed78c649f9a51e34588665574b; no voters: 8e903de616f7475ab45b506cb1b1d94a, 949919c78932479794df00fa2197fb10
I20260809 00:40:19.554983 24837 raft_consensus.cc:2726] T 27642c39bc81494aa79cea160acb0773 P 584645ed78c649f9a51e34588665574b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:40:19.719879 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:19.719628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:40241)
0809 00:40:19.719794 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:19.719831 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260809 00:40:19.793704 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:19.793474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:46119)
0809 00:40:19.793608 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:19.793653 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260809 00:40:19.929289 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:19.929013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:51343)
0809 00:40:19.929198 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:19.929241 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260809 00:40:20.720713 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:20.720468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:54925)
0809 00:40:20.720621 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:20.720662 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260809 00:40:20.794517 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:20.794291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:52685)
0809 00:40:20.794429 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:20.794471 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260809 00:40:20.930130 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:20.929779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:32793)
0809 00:40:20.930017 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:20.930055 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":94,"threads_started":1}
I20260809 00:40:21.721422 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:21.721199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:58455)
0809 00:40:21.721333 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:21.721374 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260809 00:40:21.795226 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:21.795002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:41715)
0809 00:40:21.795139 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:21.795179 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260809 00:40:21.930878 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:21.930634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:55729)
0809 00:40:21.930792 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:21.930828 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260809 00:40:22.722307 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:22.722076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.65:36525)
0809 00:40:22.722210 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:22.722254 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260809 00:40:22.796157 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:22.795844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.66:44941)
0809 00:40:22.796000 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:22.796045 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260809 00:40:22.926815 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24329
I20260809 00:40:22.931790 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:22.931547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:44961 (local address 127.7.105.67:48731)
0809 00:40:22.931683 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:22.931734 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260809 00:40:22.934633 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24506
I20260809 00:40:22.942098 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24651
2026-08-09T00:40:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18547 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:40:23 dist-test-slave-1zqn krb5kdc[24853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:40:23 dist-test-slave-1zqn krb5kdc[24853](info): set up 2 sockets
Aug 09 00:40:23 dist-test-slave-1zqn krb5kdc[24853](info): commencing operation
krb5kdc: starting...
W20260809 00:40:24.066862 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:40:24.203939 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:38803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:24.291003 24866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:24.291226 24866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:24.291249 24866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:24.292413 24866 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:40:24.292451 24866 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:40:24.292464 24866 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:40:24.292476 24866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:24.292490 24866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:24.292502 24866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:24.292514 24866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:24.292526 24866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:24.294018 24866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:38803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:24.294222 24866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:24.294418 24866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:24.300172 24871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.300211 24866 server_base.cc:1034] running on GCE node
W20260809 00:40:24.300179 24874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.300220 24872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.300726 24866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:24.301087 24866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:24.302265 24866 hybrid_clock.cc:648] HybridClock initialized: now 1786236024302252 us; error 55 us; skew 500 ppm
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:24.305114 24866 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:24.305634 24866 webserver.cc:460] Webserver started at http://127.7.105.126:36225/ using document root <none> and password file <none>
I20260809 00:40:24.305887 24866 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:24.305959 24866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:24.306128 24866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:24.307256 24866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "a97556ffb6ec4e27923a283732920d5e"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "50e07c0da19fcc162a6ff9a713147e9d"
server_key_iv: "8f5b39b9f5b85e65ad3d5b035c042622"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.307624 24866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "a97556ffb6ec4e27923a283732920d5e"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "50e07c0da19fcc162a6ff9a713147e9d"
server_key_iv: "8f5b39b9f5b85e65ad3d5b035c042622"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.309207 24866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:40:24.310153 24881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.310421 24866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:24.310557 24866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a97556ffb6ec4e27923a283732920d5e"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "50e07c0da19fcc162a6ff9a713147e9d"
server_key_iv: "8f5b39b9f5b85e65ad3d5b035c042622"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.310648 24866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:24.336627 24866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:24.338941 24866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:24.339135 24866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:24.343786 24866 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:38803
I20260809 00:40:24.343847 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:38803 every 8 connection(s)
I20260809 00:40:24.344249 24866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:24.345206 24943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:24.347541 24943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap starting.
I20260809 00:40:24.348222 24943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:24.348485 24943 log.cc:824] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:24.349195 24943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: No bootstrap required, opened a new log
I20260809 00:40:24.350157 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24866
I20260809 00:40:24.350270 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:40:24.350473 7589 external_mini_cluster.cc:1428] Setting key 7aca56278bb5e63c0045d38d393e54b7
I20260809 00:40:24.351467 24943 raft_consensus.cc:348] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:24.351644 24943 raft_consensus.cc:374] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:24.351684 24943 raft_consensus.cc:729] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Initialized, Role: FOLLOWER
I20260809 00:40:24.351872 24943 consensus_queue.cc:260] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:24.351982 24943 raft_consensus.cc:388] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:24.352020 24943 raft_consensus.cc:482] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:24.352111 24943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:24.352800 24943 raft_consensus.cc:504] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:24.352891 24943 leader_election.cc:302] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a97556ffb6ec4e27923a283732920d5e; no voters:
I20260809 00:40:24.353108 24943 leader_election.cc:288] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:40:24.353168 24948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:24.353327 24948 raft_consensus.cc:686] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 LEADER]: Becoming Leader. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Running, Role: LEADER
I20260809 00:40:24.353415 24943 sys_catalog.cc:564] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:24.353483 24948 consensus_queue.cc:237] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:24.353962 24948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a97556ffb6ec4e27923a283732920d5e. Latest consensus state: current_term: 1 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:40:24.353977 24950 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:40:24.354122 24950 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:24.354316 24948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:24.354508 24956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:24.355332 24956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:24.357216 24956 catalog_manager.cc:1347] Generated new cluster ID: 85f650e2888043b1a299c70bf8c6daa6
I20260809 00:40:24.357273 24956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:24.362855 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.351072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.7.105.126:38803)
0809 00:40:24.351304 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.351316 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.351348 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:40:24.351446 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.351451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.351494 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.351612 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:24.352486 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.354264 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.355346 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.355934 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.356047 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.359017 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:24.359021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:24.359021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:24.359033 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:24.361054 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.361488 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.361491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.361491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.361614 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.361885 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.361886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.361887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.362213 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:24.362368 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.362500 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.362647 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
I20260809 00:40:24.363370 24956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:40:24.363952 24956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:24.370344 24956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Generated new TSK 0
I20260809 00:40:24.370543 24956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:40:24.404028 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:24.489123 24969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:24.489351 24969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:24.489377 24969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:24.490614 24969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:24.490667 24969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:24.490695 24969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:40:24.492354 24969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:24.492628 24969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:24.492894 24969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:24.498590 24976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.498658 24975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.498838 24969 server_base.cc:1034] running on GCE node
W20260809 00:40:24.498648 24978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.499221 24969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:24.499511 24969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:24.500656 24969 hybrid_clock.cc:648] HybridClock initialized: now 1786236024500642 us; error 32 us; skew 500 ppm
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:24.503477 24969 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:40:24.504139 24969 webserver.cc:460] Webserver started at http://127.7.105.65:33295/ using document root <none> and password file <none>
I20260809 00:40:24.504400 24969 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:24.504496 24969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:24.504662 24969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:24.505892 24969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "b37a4fb345a14126bff4534693f0a469"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "83dda74f9972ce4e26d1d4ee653769c7"
server_key_iv: "c65710c2bc6550bfc7ecf68de8d95b37"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.506335 24969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "b37a4fb345a14126bff4534693f0a469"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "83dda74f9972ce4e26d1d4ee653769c7"
server_key_iv: "c65710c2bc6550bfc7ecf68de8d95b37"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.507874 24969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:40:24.508885 24986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.509141 24969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:24.509258 24969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "b37a4fb345a14126bff4534693f0a469"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "83dda74f9972ce4e26d1d4ee653769c7"
server_key_iv: "c65710c2bc6550bfc7ecf68de8d95b37"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.509339 24969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:24.559546 24969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:24.562090 24969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:24.562312 24969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:24.562665 24969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:24.563036 24969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:24.563097 24969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.563143 24969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:24.563177 24969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.570019 24969 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:34425
I20260809 00:40:24.570072 25110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:34425 every 8 connection(s)
I20260809 00:40:24.570495 24969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:24.580605 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 24969
I20260809 00:40:24.580721 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:40:24.581012 7589 external_mini_cluster.cc:1428] Setting key a9f78d65b358e4640cfbfec44f1d43ed
I20260809 00:40:24.581338 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.571328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53101 (local address 127.7.105.126:38803)
0809 00:40:24.571399 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.571401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.571643 (+ 242us) server_negotiation.cc:407] Connection header received
0809 00:40:24.572301 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.572302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.572316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.572383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:24.573544 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.574070 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.575198 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.575435 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.575502 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.578040 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:24.578042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:24.578042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:24.578052 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:24.579637 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.580156 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.580157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.580157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.580223 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.580514 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.580515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.580516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.580642 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:24.580733 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.581101 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.581211 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:40:24.581527 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.571272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:53101)
0809 00:40:24.571487 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.571534 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.572178 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.572473 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.572477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.573219 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:24.573441 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.573454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.574168 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.574169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.575099 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.575102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.575570 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.575571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.575691 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.576346 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:24.576356 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:24.577913 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:24.579750 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.579758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.579760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.580023 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.580310 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.580311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.580312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.580430 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.580746 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:24.580755 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:24.580935 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.581062 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.581251 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":136,"threads_started":1}
I20260809 00:40:24.582026 25111 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:24.582154 25111 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:24.582360 25111 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:24.582962 24900 ts_manager.cc:194] Registered new tserver with Master: b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:24.583582 24900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:53101
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:40:24.612385 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:24.697885 25116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:24.698125 25116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:24.698167 25116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:24.699343 25116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:24.699414 25116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:24.699463 25116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:40:24.700991 25116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:24.701259 25116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:24.701502 25116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:24.706929 25121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.707041 25122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.706931 25126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.707000 25116 server_base.cc:1034] running on GCE node
I20260809 00:40:24.707319 25116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:24.707608 25116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:24.708778 25116 hybrid_clock.cc:648] HybridClock initialized: now 1786236024708765 us; error 46 us; skew 500 ppm
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:24.711433 25116 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:40:24.711860 25116 webserver.cc:460] Webserver started at http://127.7.105.66:40591/ using document root <none> and password file <none>
I20260809 00:40:24.712241 25116 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:24.712316 25116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:24.712468 25116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:24.713617 25116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "41460b329b14402c9d422fb6215aeb3b"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "45742034ef2ecb332d7a4e232d0c224d"
server_key_iv: "50f5fa0aaad2b7fb8860e2d0ae7876c9"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.713985 25116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "41460b329b14402c9d422fb6215aeb3b"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "45742034ef2ecb332d7a4e232d0c224d"
server_key_iv: "50f5fa0aaad2b7fb8860e2d0ae7876c9"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.715420 25116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:40:24.716195 25133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.716370 25116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:24.716477 25116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "41460b329b14402c9d422fb6215aeb3b"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "45742034ef2ecb332d7a4e232d0c224d"
server_key_iv: "50f5fa0aaad2b7fb8860e2d0ae7876c9"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.716562 25116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:24.732172 25116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:24.734524 25116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:24.734699 25116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:24.734953 25116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:24.735303 25116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:24.735358 25116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.735399 25116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:24.735428 25116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.741434 25116 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39185
I20260809 00:40:24.741508 25258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39185 every 8 connection(s)
I20260809 00:40:24.741858 25116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:24.748811 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25116
I20260809 00:40:24.748934 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:40:24.749207 7589 external_mini_cluster.cc:1428] Setting key 6f5e0a1ec504e1190750640907260867
I20260809 00:40:24.752135 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.742696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:57685 (local address 127.7.105.126:38803)
0809 00:40:24.742761 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.742763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.742941 (+ 178us) server_negotiation.cc:407] Connection header received
0809 00:40:24.743418 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.743419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.743428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.743493 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:24.744530 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.745034 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.746063 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.746261 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.746328 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.748646 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:24.748648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:24.748648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:24.748656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:24.750430 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.750895 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.750897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.750897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.750975 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.751266 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.751267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.751267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.751418 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:24.751499 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.751860 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.751970 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:40:24.752302 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.742644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:57685)
0809 00:40:24.742813 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.742856 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.743336 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.743564 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.743567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.744227 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:24.744434 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.744439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.745128 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.745129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.745960 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.745963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.746365 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.746366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.746490 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.747006 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:24.747014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:24.748551 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:24.750536 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.750543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.750545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.750782 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.751089 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.751090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.751090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.751199 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.751520 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:24.751529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:24.751712 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.751815 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.751958 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260809 00:40:24.753144 25259 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:24.753269 25259 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:24.753477 25259 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:24.753916 24900 ts_manager.cc:194] Registered new tserver with Master: 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
I20260809 00:40:24.754506 24900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:57685
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:40:24.780200 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:24.875864 25264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:24.876238 25264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:24.876271 25264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:24.877514 25264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:24.877553 25264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:24.877583 25264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:40:24.879161 25264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:24.879369 25264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:24.879575 25264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:24.885335 25269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.885300 25270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:24.885300 25272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:24.885434 25264 server_base.cc:1034] running on GCE node
I20260809 00:40:24.885890 25264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:24.886188 25264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:24.887365 25264 hybrid_clock.cc:648] HybridClock initialized: now 1786236024887352 us; error 33 us; skew 500 ppm
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:24.890177 25264 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:40:24.890772 25264 webserver.cc:460] Webserver started at http://127.7.105.67:43939/ using document root <none> and password file <none>
I20260809 00:40:24.891042 25264 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:24.891093 25264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:24.891250 25264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:40:24.892467 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "7048ecdd4c444e6e9a735cdac5edbf33"
server_key_iv: "b20921618af457ea33d2bc5820a0bdc8"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.892836 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "7048ecdd4c444e6e9a735cdac5edbf33"
server_key_iv: "b20921618af457ea33d2bc5820a0bdc8"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.894320 25264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:40:24.895236 25279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.895491 25264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:24.895617 25264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "7048ecdd4c444e6e9a735cdac5edbf33"
server_key_iv: "b20921618af457ea33d2bc5820a0bdc8"
server_key_version: "encryptionkey@0"
I20260809 00:40:24.895699 25264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:24.912504 25264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:24.914914 25264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:24.915122 25264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:24.915371 25264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:24.915700 25264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:24.915755 25264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.915796 25264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:24.915825 25264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:24.922219 25264 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38441
I20260809 00:40:24.922286 25403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38441 every 8 connection(s)
I20260809 00:40:24.922652 25264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:40:24.927047 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25264
I20260809 00:40:24.927155 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:40:24.927372 7589 external_mini_cluster.cc:1428] Setting key 5a62c6f7666e6444b05976f0efc79519
Aug 09 00:40:24 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236024, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:24.932984 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.923445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:35901 (local address 127.7.105.126:38803)
0809 00:40:24.923497 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.923499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.923713 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:40:24.924280 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.924282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.924294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.924367 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:24.925356 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.925928 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.926935 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.927280 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.927372 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.930012 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:24.930014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:24.930014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:24.930023 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:24.931364 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.931852 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.931854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.931855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.931925 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.932268 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.932270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.932270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.932397 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:24.932490 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.932793 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.932885 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:40:24.933233 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.923400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:35901)
0809 00:40:24.923576 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.923620 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.924172 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.924455 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.924459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.925055 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:24.925254 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.925259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.926019 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.926020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.926834 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.926837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.927386 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.927387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.927560 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.928297 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:24.928306 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:24.929896 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:24.931503 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.931511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.931514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.931760 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.932007 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:24.932008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:24.932008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:24.932183 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:24.932481 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:24.932488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:24.932639 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.932751 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.932931 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260809 00:40:24.933760 25404 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:24.933880 25404 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:24.934073 25404 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:24.934480 24900 ts_manager.cc:194] Registered new tserver with Master: 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441)
I20260809 00:40:24.934852 24900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:35901
I20260809 00:40:24.939486 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:40:24.947207 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.941091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.7.105.126:38803)
0809 00:40:24.941160 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.941162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.941223 (+ 61us) server_negotiation.cc:407] Connection header received
0809 00:40:24.941261 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.941262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.941270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.941320 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:24.941827 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.942285 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.943243 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.943437 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.943496 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.944945 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:24.944946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:24.944947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:24.944953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:24.946198 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.946552 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.946553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.946553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.946609 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:24.946773 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:24.946773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:24.946773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:24.946914 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:24.946999 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.947082 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.947143 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:40:24.949213 24900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:40:24.950944 24900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:40:24.958506 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.953707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38441 (local address 127.0.0.1:48354)
0809 00:40:24.954184 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.954205 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.954276 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.954641 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.954644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.954647 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:24.954750 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.954752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.956193 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.956195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.957196 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.957199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.958175 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.958176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.958244 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.958256 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.958330 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.958384 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":112,"thread_start_us":90,"threads_started":1}
I20260809 00:40:24.958954 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.953876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.7.105.67:38441)
0809 00:40:24.954138 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.954143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.954364 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:40:24.954410 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.954413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.954430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.954515 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:24.954833 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.956034 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.957299 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.958065 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.958153 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.958216 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.958363 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.958412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":132,"threads_started":1}
I20260809 00:40:24.959102 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.954108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.7.105.65:34425)
0809 00:40:24.954326 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.954330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.954517 (+ 187us) server_negotiation.cc:407] Connection header received
0809 00:40:24.954676 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.954679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.954695 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.954780 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:24.954908 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.956048 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.957580 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.958397 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.958486 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.958552 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.958885 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.958940 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260809 00:40:24.959409 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.953990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.0.0.1:46184)
0809 00:40:24.954402 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.954414 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.954535 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.954790 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.954792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.954793 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:24.954831 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.954833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.956221 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.956222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.957480 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.957483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.958515 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.958516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.958587 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.958593 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.958691 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.959335 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":14,"thread_start_us":114,"threads_started":1}
I20260809 00:40:24.959985 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.953648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39185 (local address 127.0.0.1:33470)
0809 00:40:24.954158 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:24.954205 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:24.954276 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:24.954709 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:24.954710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:24.954712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:24.954788 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.954789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.955853 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.955854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.957481 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:24.957483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.959694 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.959696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:24.959799 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.959804 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:40:24.959869 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:40:24.959912 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":122,"thread_start_us":96,"threads_started":1}
I20260809 00:40:24.959897 25330 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 (DEFAULT_TABLE table=test-table [id=41cc36000217466eab4a0d0df81d3b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:24.960044 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:24.953797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.7.105.66:39185)
0809 00:40:24.954452 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:24.954457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:24.954469 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:24.954516 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:24.954518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:24.954536 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:24.954621 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:24.954884 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.955728 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.958598 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:24.959517 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:24.959598 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:24.959657 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:24.959910 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 00:40:24.959957 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":133,"threads_started":1}
I20260809 00:40:24.960338 25330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4a2e1371d4a28850040b4846dbb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:24.960266 25037 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 (DEFAULT_TABLE table=test-table [id=41cc36000217466eab4a0d0df81d3b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:24.960621 25037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4a2e1371d4a28850040b4846dbb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:24.961102 25185 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 (DEFAULT_TABLE table=test-table [id=41cc36000217466eab4a0d0df81d3b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:24.961469 25185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4a2e1371d4a28850040b4846dbb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:24.963131 25423 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Bootstrap starting.
I20260809 00:40:24.963829 25423 tablet_bootstrap.cc:654] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:24.964457 25423 log.cc:824] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:24.964967 25425 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Bootstrap starting.
I20260809 00:40:24.965379 25423 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: No bootstrap required, opened a new log
I20260809 00:40:24.965488 25426 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Bootstrap starting.
I20260809 00:40:24.965689 25423 ts_tablet_manager.cc:1397] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:40:24.965708 25425 tablet_bootstrap.cc:654] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:24.966226 25426 tablet_bootstrap.cc:654] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:24.966260 25425 log.cc:824] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:24.966954 25426 log.cc:824] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:24.967242 25425 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: No bootstrap required, opened a new log
I20260809 00:40:24.967325 25425 ts_tablet_manager.cc:1397] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:24.967898 25426 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: No bootstrap required, opened a new log
I20260809 00:40:24.967978 25426 ts_tablet_manager.cc:1397] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:40:24.968250 25423 raft_consensus.cc:348] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.968485 25423 raft_consensus.cc:374] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:24.968575 25423 raft_consensus.cc:729] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5b9bbae022415aac2ecf41a2b244ae, State: Initialized, Role: FOLLOWER
I20260809 00:40:24.968758 25423 consensus_queue.cc:260] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.969137 25423 ts_tablet_manager.cc:1428] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 00:40:24.969166 25404 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
I20260809 00:40:24.969452 25425 raft_consensus.cc:348] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.969651 25425 raft_consensus.cc:374] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:24.969741 25425 raft_consensus.cc:729] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37a4fb345a14126bff4534693f0a469, State: Initialized, Role: FOLLOWER
I20260809 00:40:24.969903 25425 consensus_queue.cc:260] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.970242 25425 ts_tablet_manager.cc:1428] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:40:24.970252 25111 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
I20260809 00:40:24.970329 25426 raft_consensus.cc:348] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.970577 25426 raft_consensus.cc:374] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:24.970641 25426 raft_consensus.cc:729] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41460b329b14402c9d422fb6215aeb3b, State: Initialized, Role: FOLLOWER
I20260809 00:40:24.970774 25426 consensus_queue.cc:260] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:24.971097 25426 ts_tablet_manager.cc:1428] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:40:24.971190 25259 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
W20260809 00:40:24.992487 25260 tablet.cc:2369] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:40:25.071473 25112 tablet.cc:2369] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:40:25.120467 25431 raft_consensus.cc:482] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:25.120671 25431 raft_consensus.cc:504] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:25.121146 25431 leader_election.cc:288] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:25.124675 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.121449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:58597 (local address 127.7.105.65:34425)
0809 00:40:25.121501 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:25.121503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:25.121515 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:40:25.121577 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:25.121578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:25.121588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:25.121641 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:25.121883 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.122503 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.123612 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.124252 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.124354 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:25.124364 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:25.124536 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 00:40:25.124588 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:40:25.124684 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.121479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:37943 (local address 127.7.105.67:38441)
0809 00:40:25.121545 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:25.121547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:25.121784 (+ 237us) server_negotiation.cc:407] Connection header received
0809 00:40:25.121816 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:25.121817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:25.121830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:25.121883 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:25.122093 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.122616 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.123836 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.124248 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.124309 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:25.124344 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:25.124569 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 00:40:25.124616 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:40:25.124684 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.121357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.7.105.66:58597)
0809 00:40:25.121440 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:25.121454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:25.121512 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:25.121659 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:25.121660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:25.121662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:25.121771 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:25.121773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.122598 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.122599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:25.123500 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:25.123503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.124376 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.124376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:25.124450 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:25.124453 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:40:25.124499 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:40:25.124560 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 00:40:25.124715 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.121364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38441 (local address 127.7.105.66:37943)
0809 00:40:25.121677 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:25.121686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:25.121743 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:25.121903 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:25.121904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:25.121906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:25.121998 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:25.122000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.122725 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.122725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:25.123728 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:25.123731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.124343 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.124344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:25.124441 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:25.124445 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:25.124550 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:40:25.124592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":94,"thread_start_us":156,"threads_started":1}
I20260809 00:40:25.125111 25352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4a2e1371d4a28850040b4846dbb53" candidate_uuid: "41460b329b14402c9d422fb6215aeb3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" is_pre_election: true
I20260809 00:40:25.125100 25058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4a2e1371d4a28850040b4846dbb53" candidate_uuid: "41460b329b14402c9d422fb6215aeb3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37a4fb345a14126bff4534693f0a469" is_pre_election: true
I20260809 00:40:25.125276 25352 raft_consensus.cc:2454] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41460b329b14402c9d422fb6215aeb3b in term 0.
I20260809 00:40:25.125285 25058 raft_consensus.cc:2454] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41460b329b14402c9d422fb6215aeb3b in term 0.
I20260809 00:40:25.125567 25135 leader_election.cc:302] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41460b329b14402c9d422fb6215aeb3b, b37a4fb345a14126bff4534693f0a469; no voters:
I20260809 00:40:25.125720 25431 raft_consensus.cc:2781] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:40:25.125782 25431 raft_consensus.cc:482] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:40:25.125805 25431 raft_consensus.cc:3037] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:25.126523 25431 raft_consensus.cc:504] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:25.126674 25431 leader_election.cc:288] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [CANDIDATE]: Term 1 election: Requested vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:25.126888 25058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4a2e1371d4a28850040b4846dbb53" candidate_uuid: "41460b329b14402c9d422fb6215aeb3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37a4fb345a14126bff4534693f0a469"
I20260809 00:40:25.126959 25058 raft_consensus.cc:3037] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:25.126855 25352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4a2e1371d4a28850040b4846dbb53" candidate_uuid: "41460b329b14402c9d422fb6215aeb3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
I20260809 00:40:25.127034 25352 raft_consensus.cc:3037] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:25.127786 25058 raft_consensus.cc:2454] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41460b329b14402c9d422fb6215aeb3b in term 1.
I20260809 00:40:25.127902 25352 raft_consensus.cc:2454] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41460b329b14402c9d422fb6215aeb3b in term 1.
I20260809 00:40:25.127979 25135 leader_election.cc:302] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41460b329b14402c9d422fb6215aeb3b, b37a4fb345a14126bff4534693f0a469; no voters:
I20260809 00:40:25.128089 25431 raft_consensus.cc:2781] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:25.128275 25431 raft_consensus.cc:686] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 1 LEADER]: Becoming Leader. State: Replica: 41460b329b14402c9d422fb6215aeb3b, State: Running, Role: LEADER
I20260809 00:40:25.128420 25431 consensus_queue.cc:237] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:25.129495 24899 catalog_manager.cc:5495] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 41460b329b14402c9d422fb6215aeb3b (127.7.105.66). New cstate: current_term: 1 leader_uuid: "41460b329b14402c9d422fb6215aeb3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } health_report { overall_health: UNKNOWN } } }
I20260809 00:40:25.142627 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.139895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.7.105.66:39185)
0809 00:40:25.139976 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:25.139978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:25.139989 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:40:25.140118 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:25.140119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:25.140145 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:25.140211 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:25.140350 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.140886 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.141858 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:25.142061 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:25.142118 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:25.142299 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:25.142400 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:40:25.142460 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:25.142518 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:40:25.142544 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:40:25.145022 25352 raft_consensus.cc:1261] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Refusing update from remote peer 41460b329b14402c9d422fb6215aeb3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:25.145062 25058 raft_consensus.cc:1261] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Refusing update from remote peer 41460b329b14402c9d422fb6215aeb3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:25.145457 25431 consensus_queue.cc:1035] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:25.145627 25433 consensus_queue.cc:1035] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:25.148150 25437 mvcc.cc:205] Tried to move back new op lower bound from 7316422758989598720 to 7316422758926823424. Current Snapshot: MvccSnapshot[applied={T|T < 7316422758989598720}]
I20260809 00:40:25.148672 25439 mvcc.cc:205] Tried to move back new op lower bound from 7316422758989598720 to 7316422758926823424. Current Snapshot: MvccSnapshot[applied={T|T < 7316422758989598720}]
I20260809 00:40:25.148816 25438 mvcc.cc:205] Tried to move back new op lower bound from 7316422758989598720 to 7316422758926823424. Current Snapshot: MvccSnapshot[applied={T|T < 7316422758989598720}]
I20260809 00:40:25.150704 24900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:25.150817 24900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:25.151690 24900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Sending DeleteTablet for 3 replicas of tablet d8e4a2e1371d4a28850040b4846dbb53
I20260809 00:40:25.152138 25185 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:25 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:33470
I20260809 00:40:25.152118 25037 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:25 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:46184
I20260809 00:40:25.152158 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24969
I20260809 00:40:25.152216 25330 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:25 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:48354
I20260809 00:40:25.156617 25448 tablet_replica.cc:330] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: stopping tablet replica
I20260809 00:40:25.156769 25448 raft_consensus.cc:2229] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:25.156925 25448 raft_consensus.cc:2258] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:25.157120 25447 tablet_replica.cc:330] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: stopping tablet replica
I20260809 00:40:25.157228 25447 raft_consensus.cc:2229] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:40:25.157377 25447 raft_consensus.cc:2258] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:25.157615 25448 ts_tablet_manager.cc:1905] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:25.159475 25448 ts_tablet_manager.cc:1918] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:25.159595 25448 log.cc:1196] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/d8e4a2e1371d4a28850040b4846dbb53
I20260809 00:40:25.160185 25448 ts_tablet_manager.cc:1939] T d8e4a2e1371d4a28850040b4846dbb53 P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting consensus metadata
I20260809 00:40:25.160277 25447 ts_tablet_manager.cc:1905] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:25.160835 24885 catalog_manager.cc:4841] TS 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): tablet d8e4a2e1371d4a28850040b4846dbb53 (table test-table [id=41cc36000217466eab4a0d0df81d3b4e]) successfully deleted
I20260809 00:40:25.162070 25447 ts_tablet_manager.cc:1918] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:25.162184 25447 log.cc:1196] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/d8e4a2e1371d4a28850040b4846dbb53
I20260809 00:40:25.162547 25447 ts_tablet_manager.cc:1939] T d8e4a2e1371d4a28850040b4846dbb53 P 41460b329b14402c9d422fb6215aeb3b: Deleting consensus metadata
W20260809 00:40:25.162709 24883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:34425 (error 108)
I20260809 00:40:25.163062 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25116
I20260809 00:40:25.163189 24883 catalog_manager.cc:4841] TS 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): tablet d8e4a2e1371d4a28850040b4846dbb53 (table test-table [id=41cc36000217466eab4a0d0df81d3b4e]) successfully deleted
I20260809 00:40:25.166682 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:25.163142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.0.0.1:46186)
0809 00:40:25.163969 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:25.166647 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":802}
W20260809 00:40:25.166854 24883 catalog_manager.cc:4584] TS b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8e4a2e1371d4a28850040b4846dbb53: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34425: connect: Connection refused (error 111)
I20260809 00:40:25.171427 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25264
I20260809 00:40:25.177245 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 24866
I20260809 00:40:45.185990 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--webserver_interface=127.7.105.126
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:38803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:40:45.271986 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:45.272312 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:45.272351 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:45.273712 25450 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:40:45.273762 25450 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:40:45.273775 25450 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:40:45.273787 25450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:40:45.273800 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:45.273813 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:45.273825 25450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:40:45.273887 25450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:40:45.275450 25450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:38803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:45.275731 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:45.276005 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:45.286010 25457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.286159 25450 server_base.cc:1034] running on GCE node
W20260809 00:40:45.286211 25456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.286073 25459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.286592 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:45.286970 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:45.288151 25450 hybrid_clock.cc:648] HybridClock initialized: now 1786236045288130 us; error 39 us; skew 500 ppm
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:45.296833 25450 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:40:45.297360 25450 webserver.cc:460] Webserver started at http://127.7.105.126:36225/ using document root <none> and password file <none>
I20260809 00:40:45.297631 25450 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:45.297715 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:45.299240 25450 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:40:45.299955 25466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.300161 25450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:40:45.300278 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a97556ffb6ec4e27923a283732920d5e"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "50e07c0da19fcc162a6ff9a713147e9d"
server_key_iv: "8f5b39b9f5b85e65ad3d5b035c042622"
server_key_version: "encryptionkey@0"
I20260809 00:40:45.300724 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:45.320602 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:45.322916 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:45.323098 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:45.327567 25450 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:38803
I20260809 00:40:45.327610 25522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:38803 every 8 connection(s)
I20260809 00:40:45.328096 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:40:45.329698 25523 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:40:45.330346 25523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap starting.
I20260809 00:40:45.332705 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25450
I20260809 00:40:45.333058 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:34425
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=33295
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:45.340691 25523 log.cc:824] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:45.342921 25523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:45.343268 25523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap complete.
I20260809 00:40:45.345749 25523 raft_consensus.cc:348] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:45.345980 25523 raft_consensus.cc:729] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Initialized, Role: FOLLOWER
I20260809 00:40:45.346161 25523 consensus_queue.cc:260] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:45.346232 25523 raft_consensus.cc:388] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:40:45.346302 25523 raft_consensus.cc:482] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:40:45.346356 25523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:40:45.347438 25523 raft_consensus.cc:504] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:45.347580 25523 leader_election.cc:302] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a97556ffb6ec4e27923a283732920d5e; no voters:
I20260809 00:40:45.347803 25523 leader_election.cc:288] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:40:45.347846 25527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:40:45.347975 25527 raft_consensus.cc:686] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 LEADER]: Becoming Leader. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Running, Role: LEADER
I20260809 00:40:45.348112 25523 sys_catalog.cc:564] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:40:45.348083 25527 consensus_queue.cc:237] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:40:45.348409 25530 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a97556ffb6ec4e27923a283732920d5e. Latest consensus state: current_term: 2 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:40:45.348560 25528 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:40:45.348654 25528 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:45.348623 25530 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:40:45.348933 25533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:40:45.349843 25533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:40:45.350009 25533 catalog_manager.cc:1259] Loaded cluster ID: 85f650e2888043b1a299c70bf8c6daa6
I20260809 00:40:45.350199 25533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:40:45.351400 25533 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:40:45.351580 25533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Loaded TSK: 0
I20260809 00:40:45.374769 25533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Generated new TSK 1
I20260809 00:40:45.374897 25533 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:40:45.420346 25525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:45.420594 25525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:45.420639 25525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:45.421957 25525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:45.422035 25525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:45.422083 25525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:40:45.423593 25525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:34425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=33295
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:45.423877 25525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:45.424172 25525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:45.430737 25554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.430812 25552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.430751 25551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.431005 25525 server_base.cc:1034] running on GCE node
I20260809 00:40:45.431597 25525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:45.431922 25525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:45.433126 25525 hybrid_clock.cc:648] HybridClock initialized: now 1786236045433095 us; error 46 us; skew 500 ppm
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:45.436363 25525 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:40:45.436882 25525 webserver.cc:460] Webserver started at http://127.7.105.65:33295/ using document root <none> and password file <none>
I20260809 00:40:45.437165 25525 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:45.437256 25525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:45.438771 25525 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:40:45.439574 25563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.439832 25525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.439937 25525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "b37a4fb345a14126bff4534693f0a469"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "83dda74f9972ce4e26d1d4ee653769c7"
server_key_iv: "c65710c2bc6550bfc7ecf68de8d95b37"
server_key_version: "encryptionkey@0"
I20260809 00:40:45.440418 25525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:45.457998 25525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:45.460522 25525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:45.460702 25525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:45.460964 25525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:45.461501 25570 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:40:45.463004 25525 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:40:45.463057 25525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:40:45.463097 25525 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:40:45.463768 25525 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:40:45.463814 25525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:40:45.463861 25570 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Bootstrap starting.
I20260809 00:40:45.471720 25525 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:34425
I20260809 00:40:45.471923 25688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:34425 every 8 connection(s)
I20260809 00:40:45.472292 25525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:40:45.475334 25570 log.cc:824] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:45.477830 25570 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:40:45.478192 25570 tablet_bootstrap.cc:492] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Bootstrap complete.
I20260809 00:40:45.479486 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25525
I20260809 00:40:45.479507 25570 ts_tablet_manager.cc:1397] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260809 00:40:45.479786 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:39185
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=40591
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:45.481325 25570 raft_consensus.cc:348] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.481739 25570 raft_consensus.cc:729] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37a4fb345a14126bff4534693f0a469, State: Initialized, Role: FOLLOWER
I20260809 00:40:45.481889 25570 consensus_queue.cc:260] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.482224 25570 ts_tablet_manager.cc:1428] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:45.488226 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.473208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:55193 (local address 127.7.105.126:38803)
0809 00:40:45.473463 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.473474 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.475788 (+ 2314us) server_negotiation.cc:407] Connection header received
0809 00:40:45.476524 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.476532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.476594 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.476764 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.477582 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.479149 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.481024 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.481647 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.481829 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.484621 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.484626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.484626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.484639 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.486381 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.486839 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.486843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.486843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.486940 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.487228 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.487229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.487230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.487458 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.487551 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.487865 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.487993 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260809 00:40:45.488227 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.473066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:55193)
0809 00:40:45.475563 (+ 2497us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.475622 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.476277 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.476809 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.476813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.477169 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:45.477438 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.477444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.479297 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.479299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.480915 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.480918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.481947 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.481948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.482112 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.482769 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:45.482779 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:45.484492 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:45.486494 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.486503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.486505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.486741 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.487021 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.487022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.487023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.487151 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.487543 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:45.487551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:45.487694 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.487796 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.487983 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2445,"thread_start_us":93,"threads_started":1}
I20260809 00:40:45.489279 25689 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:45.489378 25689 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:45.489560 25689 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:45.490248 25483 ts_manager.cc:194] Registered new tserver with Master: b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:45.491360 25483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:55193
I20260809 00:40:45.492444 25689 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.65@KRBTEST.COM
I20260809 00:40:45.499845 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.490855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.0.0.1:41534)
0809 00:40:45.491693 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.491728 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.491790 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.492114 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.492120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.492437 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:45.492501 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.492503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.493663 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.493664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.494282 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.494285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.494770 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.494772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.494850 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.495358 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:45.495364 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:45.496759 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:45.498643 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.498651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.498653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.498852 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.499103 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.499104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.499104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.499169 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.499458 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:45.499465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:45.499517 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.499622 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.499747 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":95,"threads_started":1}
I20260809 00:40:45.499847 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.491112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.7.105.65:34425)
0809 00:40:45.491913 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.491917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.491925 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:45.491956 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.491957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.492000 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.492598 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.492650 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.493518 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.494389 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.494689 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.494772 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.496866 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.496874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.496875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.496881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.498513 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.498954 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.498957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.498958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.499018 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.499240 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.499241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.499241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.499375 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.499467 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.499659 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.499747 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":68,"threads_started":1}
I20260809 00:40:45.500634 25616 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e4a2e1371d4a28850040b4846dbb53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:25 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:41534
I20260809 00:40:45.501298 25698 tablet_replica.cc:330] stopping tablet replica
I20260809 00:40:45.501425 25698 raft_consensus.cc:2229] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:45.501518 25698 raft_consensus.cc:2258] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:45.501842 25698 ts_tablet_manager.cc:1905] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:45.504215 25698 ts_tablet_manager.cc:1918] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:45.504309 25698 log.cc:1196] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal/wals/d8e4a2e1371d4a28850040b4846dbb53
I20260809 00:40:45.504613 25698 ts_tablet_manager.cc:1939] T d8e4a2e1371d4a28850040b4846dbb53 P b37a4fb345a14126bff4534693f0a469: Deleting consensus metadata
I20260809 00:40:45.505220 25467 catalog_manager.cc:4841] TS b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425): tablet d8e4a2e1371d4a28850040b4846dbb53 (table test-table [id=41cc36000217466eab4a0d0df81d3b4e]) successfully deleted
W20260809 00:40:45.570988 25694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:45.571197 25694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:45.571221 25694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:45.572414 25694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:45.572459 25694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:45.572485 25694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:40:45.574090 25694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=40591
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:45.574350 25694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:45.574581 25694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:45.579793 25705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.579856 25704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.579990 25694 server_base.cc:1034] running on GCE node
W20260809 00:40:45.580112 25707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.580310 25694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:45.580515 25694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:45.581677 25694 hybrid_clock.cc:648] HybridClock initialized: now 1786236045581673 us; error 40 us; skew 500 ppm
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:45.584250 25694 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:40:45.584638 25694 webserver.cc:460] Webserver started at http://127.7.105.66:40591/ using document root <none> and password file <none>
I20260809 00:40:45.584834 25694 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:45.584885 25694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:45.586198 25694 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.586769 25717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.586941 25694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.587031 25694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "41460b329b14402c9d422fb6215aeb3b"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "45742034ef2ecb332d7a4e232d0c224d"
server_key_iv: "50f5fa0aaad2b7fb8860e2d0ae7876c9"
server_key_version: "encryptionkey@0"
I20260809 00:40:45.587381 25694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:45.610074 25694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:45.612026 25694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:45.612231 25694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:45.612488 25694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:45.612830 25694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:45.612865 25694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.612915 25694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:45.612946 25694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.618985 25694 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39185
I20260809 00:40:45.619068 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39185 every 8 connection(s)
I20260809 00:40:45.619448 25694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:45.627786 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25694
I20260809 00:40:45.628146 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:38441
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=43939
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:40:45.628875 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.620228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41683 (local address 127.7.105.126:38803)
0809 00:40:45.620321 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.620324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.620532 (+ 208us) server_negotiation.cc:407] Connection header received
0809 00:40:45.621008 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.621010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.621022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.621072 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.622133 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.622650 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.623571 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.623770 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.623820 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.626071 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.626072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.626072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.626078 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.627305 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.627751 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.627751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.627752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.627803 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.628136 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.628136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.628137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.628297 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.628372 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.628684 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.628780 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:40:45.629060 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.620214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:41683)
0809 00:40:45.620399 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.620435 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.620917 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.621135 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.621138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.621841 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:45.622042 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.622047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.622731 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.622732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.623497 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.623499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.623842 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.623843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.623938 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.624513 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:45.624522 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:45.625984 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:45.627377 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.627383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.627385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.627671 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.627873 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.627874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.627874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.628024 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.628380 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:45.628388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:45.628532 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.628650 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.628817 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
I20260809 00:40:45.629650 25841 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:45.629760 25841 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:45.629987 25841 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:45.630462 25483 ts_manager.cc:194] Registered new tserver with Master: 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
I20260809 00:40:45.630854 25483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:41683
W20260809 00:40:45.714129 25844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:40:45.714347 25844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:40:45.714373 25844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:40:45.715472 25844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:40:45.715544 25844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:40:45.715576 25844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:40:45.717104 25844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:38441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=43939
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:40:45.717315 25844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:40:45.717514 25844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:40:45.722841 25850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.722837 25852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:40:45.722854 25849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:40:45.723170 25844 server_base.cc:1034] running on GCE node
I20260809 00:40:45.723435 25844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:40:45.723640 25844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:40:45.725018 25844 hybrid_clock.cc:648] HybridClock initialized: now 1786236045724991 us; error 36 us; skew 500 ppm
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:40:45.727526 25844 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:40:45.727977 25844 webserver.cc:460] Webserver started at http://127.7.105.67:43939/ using document root <none> and password file <none>
I20260809 00:40:45.728245 25844 fs_manager.cc:359] Metadata directory not provided
I20260809 00:40:45.728304 25844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:40:45.729604 25844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.730245 25859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.730439 25844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.730525 25844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "7048ecdd4c444e6e9a735cdac5edbf33"
server_key_iv: "b20921618af457ea33d2bc5820a0bdc8"
server_key_version: "encryptionkey@0"
I20260809 00:40:45.730849 25844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:40:45.760617 25844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:40:45.762735 25844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:40:45.762859 25844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:40:45.763067 25844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:40:45.763422 25844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:40:45.763458 25844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.763506 25844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:40:45.763535 25844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:40:45.769544 25844 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38441
I20260809 00:40:45.769618 25982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38441 every 8 connection(s)
I20260809 00:40:45.769994 25844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:40:45.775070 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 25844
Aug 09 00:40:45 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:40:45.779662 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.770783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:57455 (local address 127.7.105.126:38803)
0809 00:40:45.770874 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.770876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.771054 (+ 178us) server_negotiation.cc:407] Connection header received
0809 00:40:45.771529 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.771530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.771541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.771592 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.772527 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.773021 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.773975 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.774157 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.774210 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.776692 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.776694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.776695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.776701 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.778127 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.778671 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.778673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.778673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.778743 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.779021 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.779022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.779023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.779177 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.779259 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.779503 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.779585 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:40:45.779978 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.770740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:57455)
0809 00:40:45.770927 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.770965 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.771445 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.771657 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.771660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.772255 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:40:45.772453 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.772458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.773107 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.773108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.773900 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.773902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.774231 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.774232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.774335 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.774880 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:40:45.774889 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:40:45.776589 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:40:45.778229 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.778236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.778239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.778574 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.778841 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:40:45.778842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:40:45.778842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:40:45.778947 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:40:45.779250 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:40:45.779254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:40:45.779366 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.779449 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.779639 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260809 00:40:45.780773 25983 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38803
I20260809 00:40:45.780870 25983 heartbeater.cc:464] Registering TS with master...
I20260809 00:40:45.781071 25983 heartbeater.cc:511] Master 127.7.105.126:38803 requested a full tablet report, sending...
I20260809 00:40:45.781493 25483 ts_manager.cc:194] Registered new tserver with Master: 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441)
I20260809 00:40:45.781865 25483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:57455
I20260809 00:40:45.782423 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.775526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.7.105.126:38803)
0809 00:40:45.775654 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.775657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.775752 (+ 95us) server_negotiation.cc:407] Connection header received
0809 00:40:45.775778 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.775779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.775791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.775867 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.776633 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.777099 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.777856 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.778054 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.778129 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.779543 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.779544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.779545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.779556 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.781232 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.781606 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.781607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.781607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.781666 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.781882 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.781883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.781883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.782009 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.782193 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.782231 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.782314 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260809 00:40:45.783191 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:40:45.785734 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.783502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.7.105.126:38803)
0809 00:40:45.783537 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.783539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.783719 (+ 180us) server_negotiation.cc:407] Connection header received
0809 00:40:45.783778 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.783779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.783806 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.783861 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:45.783929 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.784445 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.785167 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.785345 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.785396 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.785518 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:45.785555 (+ 37us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 00:40:45.785670 (+ 115us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33958: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:40:45.785791 25987 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33958: token expired
W20260809 00:40:45.785985 25408 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.105.126:38803)
I20260809 00:40:45.791675 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.786211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.7.105.126:38803)
0809 00:40:45.786270 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.786272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.786279 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:40:45.786295 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.786296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.786302 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.786344 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:40:45.786763 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.787213 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.787857 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.788020 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.788117 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.789163 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:40:45.789164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:40:45.789164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:40:45.789170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:40:45.790556 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.790851 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.790852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.790852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.790900 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:40:45.791158 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:40:45.791160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:40:45.791160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:40:45.791265 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:40:45.791373 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.791475 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.791547 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:40:45.792876 25483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:40:45.794417 25483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:40:45.796813 25616 tablet_service.cc:1467] Processing CreateTablet for tablet c981f737b3594fc09908007389d68e2a (DEFAULT_TABLE table=test-table [id=c55a6e593d3642979a786c986c868862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:45.797047 25616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c981f737b3594fc09908007389d68e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:45.798659 25570 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Bootstrap starting.
I20260809 00:40:45.798995 25570 tablet_bootstrap.cc:654] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:45.799575 25570 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: No bootstrap required, opened a new log
I20260809 00:40:45.799633 25570 ts_tablet_manager.cc:1397] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.799957 25570 raft_consensus.cc:348] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.800036 25570 raft_consensus.cc:374] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:45.800107 25570 raft_consensus.cc:729] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37a4fb345a14126bff4534693f0a469, State: Initialized, Role: FOLLOWER
I20260809 00:40:45.800163 25570 consensus_queue.cc:260] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.800258 25570 ts_tablet_manager.cc:1428] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:40:45.800460 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.796328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39185 (local address 127.0.0.1:36652)
0809 00:40:45.796548 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.796560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.796612 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.797087 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.797088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.797090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:45.797143 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.797144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.798307 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.798308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.799354 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.799356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.800144 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.800145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.800220 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.800275 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.800335 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.800389 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":44,"thread_start_us":100,"threads_started":1}
I20260809 00:40:45.800761 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.796664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.7.105.66:39185)
0809 00:40:45.796858 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.796863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.796874 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:40:45.796957 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.796960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.796976 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.797065 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:45.797201 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.798222 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.799444 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.800057 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.800144 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.800208 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.800630 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.800675 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260809 00:40:45.801483 25769 tablet_service.cc:1467] Processing CreateTablet for tablet c981f737b3594fc09908007389d68e2a (DEFAULT_TABLE table=test-table [id=c55a6e593d3642979a786c986c868862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:45.801844 25769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c981f737b3594fc09908007389d68e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:45.802248 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.796327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38441 (local address 127.0.0.1:55870)
0809 00:40:45.796503 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.796532 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.796592 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.798721 (+ 2129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.798722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.798725 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:45.798767 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.798769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.800119 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.800120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.801201 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.801204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.802067 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.802068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.802131 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.802134 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.802172 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.802202 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 00:40:45.802333 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.796451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.7.105.67:38441)
0809 00:40:45.796737 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.796742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.796753 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:40:45.796794 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.796796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.796813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.796869 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:45.798855 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.799977 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.801274 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.801985 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.802063 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.802130 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.802196 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.802247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
I20260809 00:40:45.802917 25915 tablet_service.cc:1467] Processing CreateTablet for tablet c981f737b3594fc09908007389d68e2a (DEFAULT_TABLE table=test-table [id=c55a6e593d3642979a786c986c868862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:40:45.803162 25915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c981f737b3594fc09908007389d68e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:40:45.805135 25996 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Bootstrap starting.
I20260809 00:40:45.805856 25996 tablet_bootstrap.cc:654] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:45.806361 25997 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Bootstrap starting.
I20260809 00:40:45.806502 25996 log.cc:824] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:45.807075 25997 tablet_bootstrap.cc:654] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:40:45.807238 25996 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: No bootstrap required, opened a new log
I20260809 00:40:45.807293 25996 ts_tablet_manager.cc:1397] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:45.807660 25997 log.cc:824] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Log is configured to *not* fsync() on all Append() calls
I20260809 00:40:45.808413 25997 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: No bootstrap required, opened a new log
I20260809 00:40:45.808470 25997 ts_tablet_manager.cc:1397] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:40:45.808998 25996 raft_consensus.cc:348] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.809182 25996 raft_consensus.cc:374] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:45.809232 25996 raft_consensus.cc:729] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41460b329b14402c9d422fb6215aeb3b, State: Initialized, Role: FOLLOWER
I20260809 00:40:45.809360 25996 consensus_queue.cc:260] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.809672 25996 ts_tablet_manager.cc:1428] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:40:45.809720 25841 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
I20260809 00:40:45.810201 25997 raft_consensus.cc:348] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.810374 25997 raft_consensus.cc:374] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:40:45.810420 25997 raft_consensus.cc:729] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5b9bbae022415aac2ecf41a2b244ae, State: Initialized, Role: FOLLOWER
I20260809 00:40:45.810529 25997 consensus_queue.cc:260] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.810825 25997 ts_tablet_manager.cc:1428] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:40:45.810849 25983 heartbeater.cc:503] Master 127.7.105.126:38803 was elected leader, sending a full tablet report...
I20260809 00:40:45.862118 26002 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:40:45.862257 26002 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.862686 26002 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185), b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:45.865677 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.863052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:44323 (local address 127.7.105.66:39185)
0809 00:40:45.863102 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.863104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.863113 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:40:45.863224 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.863225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.863235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.863295 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:45.863474 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.863960 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.864918 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.865298 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.865398 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.865407 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.865568 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.865614 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:40:45.865677 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.862956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39185 (local address 127.7.105.67:44323)
0809 00:40:45.863027 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.863042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.863137 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.863318 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.863319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.863320 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:45.863396 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.863398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.864046 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.864048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.864816 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.864818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.865397 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.865398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.865481 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.865484 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.865559 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.865599 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260809 00:40:45.865943 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.863078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.7.105.67:40585)
0809 00:40:45.863262 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.863271 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:40:45.863356 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:40:45.863545 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:40:45.863546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:40:45.863548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:40:45.863626 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.863628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.864315 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.864316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.865050 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:40:45.865052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.865705 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.865706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:40:45.865768 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.865772 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:40:45.865811 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 00:40:45.865848 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":20,"thread_start_us":106,"threads_started":1}
I20260809 00:40:45.865947 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.863185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:40585 (local address 127.7.105.65:34425)
0809 00:40:45.863242 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.863244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.863341 (+ 97us) server_negotiation.cc:407] Connection header received
0809 00:40:45.863426 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.863427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.863459 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.863541 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:40:45.863708 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.864236 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.865154 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.865612 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.865706 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.865792 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.865826 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.865864 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:40:45.866350 25792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c981f737b3594fc09908007389d68e2a" candidate_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41460b329b14402c9d422fb6215aeb3b" is_pre_election: true
I20260809 00:40:45.866528 25792 raft_consensus.cc:2454] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5b9bbae022415aac2ecf41a2b244ae in term 0.
I20260809 00:40:45.866788 25860 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae; no voters:
I20260809 00:40:45.866863 25637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c981f737b3594fc09908007389d68e2a" candidate_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37a4fb345a14126bff4534693f0a469" is_pre_election: true
I20260809 00:40:45.866964 26002 raft_consensus.cc:2781] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:40:45.867017 25637 raft_consensus.cc:2454] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5b9bbae022415aac2ecf41a2b244ae in term 0.
I20260809 00:40:45.867043 26002 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:40:45.867069 26002 raft_consensus.cc:3037] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:45.867909 26002 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:40:45.868073 26002 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [CANDIDATE]: Term 1 election: Requested vote from peers 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185), b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425)
I20260809 00:40:45.868259 25792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c981f737b3594fc09908007389d68e2a" candidate_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41460b329b14402c9d422fb6215aeb3b"
I20260809 00:40:45.868312 25637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c981f737b3594fc09908007389d68e2a" candidate_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37a4fb345a14126bff4534693f0a469"
I20260809 00:40:45.868341 25792 raft_consensus.cc:3037] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:45.868417 25637 raft_consensus.cc:3037] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:40:45.869053 25792 raft_consensus.cc:2454] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5b9bbae022415aac2ecf41a2b244ae in term 1.
I20260809 00:40:45.869203 25637 raft_consensus.cc:2454] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5b9bbae022415aac2ecf41a2b244ae in term 1.
I20260809 00:40:45.869228 25860 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae; no voters:
I20260809 00:40:45.869361 26002 raft_consensus.cc:2781] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:40:45.869537 26002 raft_consensus.cc:686] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 LEADER]: Becoming Leader. State: Replica: 5c5b9bbae022415aac2ecf41a2b244ae, State: Running, Role: LEADER
I20260809 00:40:45.869649 26002 consensus_queue.cc:237] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
W20260809 00:40:45.869994 25842 tablet.cc:2369] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:40:45.870543 25482 catalog_manager.cc:5495] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67). New cstate: current_term: 1 leader_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } health_report { overall_health: UNKNOWN } } }
I20260809 00:40:45.876456 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.874246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.7.105.67:38441)
0809 00:40:45.874307 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:40:45.874309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:40:45.874317 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:40:45.874354 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:40:45.874355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:40:45.874371 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:40:45.874429 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:40:45.874538 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.874986 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.875672 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:40:45.875852 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:40:45.875904 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:40:45.876130 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:40:45.876239 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:40:45.876321 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:40:45.876359 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:40:45.876378 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:40:45.878206 25792 raft_consensus.cc:1261] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Refusing update from remote peer 5c5b9bbae022415aac2ecf41a2b244ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:45.878302 25637 raft_consensus.cc:1261] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Refusing update from remote peer 5c5b9bbae022415aac2ecf41a2b244ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:40:45.878583 26004 consensus_queue.cc:1035] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:45.878751 26002 consensus_queue.cc:1035] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:40:45.881628 26009 mvcc.cc:205] Tried to move back new op lower bound from 7316422843914674176 to 7316422843882844160. Current Snapshot: MvccSnapshot[applied={T|T < 7316422843914674176}]
I20260809 00:40:45.883534 25483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:45.883636 25483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:40:45.884387 25483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Sending DeleteTablet for 3 replicas of tablet c981f737b3594fc09908007389d68e2a
I20260809 00:40:45.884744 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25525
I20260809 00:40:45.884779 25769 tablet_service.cc:1514] Processing DeleteTablet for tablet c981f737b3594fc09908007389d68e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:45 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:36652
I20260809 00:40:45.884825 25915 tablet_service.cc:1514] Processing DeleteTablet for tablet c981f737b3594fc09908007389d68e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:40:45 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:55870
I20260809 00:40:45.887076 26018 tablet_replica.cc:330] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: stopping tablet replica
I20260809 00:40:45.887188 26018 raft_consensus.cc:2229] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:40:45.887292 26018 raft_consensus.cc:2258] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:40:45.887795 26018 ts_tablet_manager.cc:1905] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:45.889863 26018 ts_tablet_manager.cc:1918] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:45.889945 26018 log.cc:1196] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/c981f737b3594fc09908007389d68e2a
I20260809 00:40:45.890252 26018 ts_tablet_manager.cc:1939] T c981f737b3594fc09908007389d68e2a P 41460b329b14402c9d422fb6215aeb3b: Deleting consensus metadata
I20260809 00:40:45.890767 25467 catalog_manager.cc:4841] TS 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): tablet c981f737b3594fc09908007389d68e2a (table test-table [id=c55a6e593d3642979a786c986c868862]) successfully deleted
W20260809 00:40:45.894488 25467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:34425 (error 108)
I20260809 00:40:45.894781 26019 tablet_replica.cc:330] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: stopping tablet replica
I20260809 00:40:45.894819 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25694
I20260809 00:40:45.895004 26019 raft_consensus.cc:2229] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:40:45.895414 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:40:45.894906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34425 (local address 127.0.0.1:41538)
0809 00:40:45.895339 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 00:40:45.895378 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413}
I20260809 00:40:45.895850 26019 raft_consensus.cc:2258] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 00:40:45.895525 25467 catalog_manager.cc:4584] TS b37a4fb345a14126bff4534693f0a469 (127.7.105.65:34425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c981f737b3594fc09908007389d68e2a: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34425: connect: Connection refused (error 111)
I20260809 00:40:45.897693 26019 ts_tablet_manager.cc:1905] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:40:45.899518 26019 ts_tablet_manager.cc:1918] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:40:45.899623 26019 log.cc:1196] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/c981f737b3594fc09908007389d68e2a
I20260809 00:40:45.899888 26019 ts_tablet_manager.cc:1939] T c981f737b3594fc09908007389d68e2a P 5c5b9bbae022415aac2ecf41a2b244ae: Deleting consensus metadata
I20260809 00:40:45.900517 25469 catalog_manager.cc:4841] TS 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): tablet c981f737b3594fc09908007389d68e2a (table test-table [id=c55a6e593d3642979a786c986c868862]) successfully deleted
I20260809 00:40:45.902066 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25844
I20260809 00:40:45.908994 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 25450
I20260809 00:41:05.916836 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--webserver_interface=127.7.105.126
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:38803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:06.007135 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:06.007370 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:06.007395 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:06.008719 26021 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:41:06.008770 26021 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:41:06.008785 26021 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:41:06.008802 26021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:06.008817 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:06.008832 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:06.008845 26021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:06.008859 26021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:06.010496 26021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:38803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:06.010725 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:06.010927 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:06.016556 26029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.016633 26027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.016655 26026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.016749 26021 server_base.cc:1034] running on GCE node
I20260809 00:41:06.017343 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:06.017619 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:06.018803 26021 hybrid_clock.cc:648] HybridClock initialized: now 1786236066018794 us; error 38 us; skew 500 ppm
Aug 09 00:41:06 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:06.024904 26021 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:06.025378 26021 webserver.cc:460] Webserver started at http://127.7.105.126:36225/ using document root <none> and password file <none>
I20260809 00:41:06.025641 26021 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:06.025714 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:06.027120 26021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:06.027818 26038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.027997 26021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:06.028133 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a97556ffb6ec4e27923a283732920d5e"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "50e07c0da19fcc162a6ff9a713147e9d"
server_key_iv: "8f5b39b9f5b85e65ad3d5b035c042622"
server_key_version: "encryptionkey@0"
I20260809 00:41:06.028512 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:06.051543 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:06.054167 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:06.054345 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:06.058674 26021 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:38803
I20260809 00:41:06.058737 26094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:38803 every 8 connection(s)
I20260809 00:41:06.059154 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:06.060899 26095 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:41:06.061645 26095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap starting.
I20260809 00:41:06.063925 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26021
I20260809 00:41:06.064316 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:34425
--local_ip_for_outbound_sockets=127.7.105.65
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=33295
--webserver_interface=127.7.105.65
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:41:06.071990 26095 log.cc:824] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:06.074628 26095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:41:06.074988 26095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Bootstrap complete.
I20260809 00:41:06.077455 26095 raft_consensus.cc:348] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:41:06.077728 26095 raft_consensus.cc:729] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Initialized, Role: FOLLOWER
I20260809 00:41:06.077919 26095 consensus_queue.cc:260] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:41:06.077998 26095 raft_consensus.cc:388] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:06.078054 26095 raft_consensus.cc:482] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:06.078106 26095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:41:06.079224 26095 raft_consensus.cc:504] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:41:06.079360 26095 leader_election.cc:302] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a97556ffb6ec4e27923a283732920d5e; no voters:
I20260809 00:41:06.079584 26095 leader_election.cc:288] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:41:06.079624 26099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:41:06.079752 26099 raft_consensus.cc:686] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [term 3 LEADER]: Becoming Leader. State: Replica: a97556ffb6ec4e27923a283732920d5e, State: Running, Role: LEADER
I20260809 00:41:06.079888 26095 sys_catalog.cc:564] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:06.079905 26099 consensus_queue.cc:237] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } }
I20260809 00:41:06.080432 26101 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a97556ffb6ec4e27923a283732920d5e. Latest consensus state: current_term: 3 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:41:06.080451 26100 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a97556ffb6ec4e27923a283732920d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97556ffb6ec4e27923a283732920d5e" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38803 } } }
I20260809 00:41:06.080533 26101 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:06.080565 26100 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:06.081163 26106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:06.082113 26106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:06.082257 26106 catalog_manager.cc:1259] Loaded cluster ID: 85f650e2888043b1a299c70bf8c6daa6
I20260809 00:41:06.082360 26106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:06.083828 26106 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:06.084128 26106 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Loaded TSK: 0
I20260809 00:41:06.084210 26106 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a97556ffb6ec4e27923a283732920d5e: Loaded TSK: 1
F20260809 00:41:06.084491 26106 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5bd17fb9ad google::LogMessage::Fail() at ??:0
@ 0x7f5bd17fe88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5bd17fb4de google::LogMessage::Flush() at ??:0
@ 0x7f5bd17fca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5bd3f68104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5bd3f65d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5bd97551cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5bd975548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5bd9755594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f5bd971bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f5bd9725ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5bd36c90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5bd36be954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5bd45226db start_thread at ??:0
@ 0x7f5bd02f961f clone at ??:0
W20260809 00:41:06.172596 26097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:06.172806 26097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:06.172832 26097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:06.174016 26097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:06.174067 26097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:06.174095 26097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:06.175591 26097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:34425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=33295
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:06.175850 26097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:06.176175 26097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:06.181617 26137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.181684 26138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.181648 26140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.182389 26097 server_base.cc:1034] running on GCE node
I20260809 00:41:06.182683 26097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:06.182958 26097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:06.184134 26097 hybrid_clock.cc:648] HybridClock initialized: now 1786236066184104 us; error 45 us; skew 500 ppm
Aug 09 00:41:06 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:06.187151 26097 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:41:06.187618 26097 webserver.cc:460] Webserver started at http://127.7.105.65:33295/ using document root <none> and password file <none>
I20260809 00:41:06.187883 26097 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:06.187955 26097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:06.189502 26097 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:06.190294 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.190505 26097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:06.190622 26097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "b37a4fb345a14126bff4534693f0a469"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "83dda74f9972ce4e26d1d4ee653769c7"
server_key_iv: "c65710c2bc6550bfc7ecf68de8d95b37"
server_key_version: "encryptionkey@0"
I20260809 00:41:06.190982 26097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:06.216912 26097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:06.219377 26097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:06.219558 26097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:06.219810 26097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:06.220367 26155 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:41:06.222599 26097 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:41:06.222666 26097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:41:06.222712 26097 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:41:06.223398 26097 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:41:06.223455 26097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:06.223577 26155 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Bootstrap starting.
I20260809 00:41:06.230921 26097 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:34425
I20260809 00:41:06.230984 26273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:34425 every 8 connection(s)
I20260809 00:41:06.231390 26097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:06.235991 26155 log.cc:824] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:06.238140 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26097
I20260809 00:41:06.238440 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:39185
--local_ip_for_outbound_sockets=127.7.105.66
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=40591
--webserver_interface=127.7.105.66
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:41:06.239053 26155 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:41:06.240224 26155 tablet_bootstrap.cc:492] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Bootstrap complete.
I20260809 00:41:06.241765 26155 ts_tablet_manager.cc:1397] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.017s
I20260809 00:41:06.243852 26155 raft_consensus.cc:348] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:06.244346 26155 raft_consensus.cc:729] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37a4fb345a14126bff4534693f0a469, State: Initialized, Role: FOLLOWER
I20260809 00:41:06.244581 26155 consensus_queue.cc:260] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:06.244935 26155 ts_tablet_manager.cc:1428] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260809 00:41:06.305016 26277 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:06.232329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:49305)
0809 00:41:06.232546 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.232592 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:06.233038 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:06.304773 (+ 71735us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":130,"threads_started":1}
W20260809 00:41:06.305258 26275 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:38803 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:38803: Client connection negotiation failed: client connection to 127.7.105.126:38803: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:41:06.305626 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.305492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:36839)
0809 00:41:06.305547 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.305598 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:41:06.306017 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.305928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:40407)
0809 00:41:06.305978 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.306002 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 00:41:06.306154 26275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:41:06.329026 26279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:06.329226 26279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:06.329249 26279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:06.330394 26279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:06.330435 26279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:06.330487 26279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:06.331947 26279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=40591
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:06.332216 26279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:06.332450 26279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:06.337746 26286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.337775 26289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.337755 26279 server_base.cc:1034] running on GCE node
W20260809 00:41:06.337775 26285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.338158 26279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:06.338383 26279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:06.339560 26279 hybrid_clock.cc:648] HybridClock initialized: now 1786236066339541 us; error 26 us; skew 500 ppm
Aug 09 00:41:06 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:06.342140 26279 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:41:06.342593 26279 webserver.cc:460] Webserver started at http://127.7.105.66:40591/ using document root <none> and password file <none>
I20260809 00:41:06.342837 26279 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:06.342908 26279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:06.344347 26279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:06.345011 26296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.345199 26279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:06.345309 26279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "41460b329b14402c9d422fb6215aeb3b"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "45742034ef2ecb332d7a4e232d0c224d"
server_key_iv: "50f5fa0aaad2b7fb8860e2d0ae7876c9"
server_key_version: "encryptionkey@0"
I20260809 00:41:06.345662 26279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:06.360829 26279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:06.362802 26279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:06.362987 26279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:06.363262 26279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:06.363622 26279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:06.363685 26279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.363723 26279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:06.363752 26279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.370133 26279 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39185
I20260809 00:41:06.370189 26419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39185 every 8 connection(s)
I20260809 00:41:06.370560 26279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:41:06.371814 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.371311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:58327)
0809 00:41:06.371587 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.371664 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
W20260809 00:41:06.372376 26420 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:38803 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:38803: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
I20260809 00:41:06.372718 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.372620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:52553)
0809 00:41:06.372669 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.372699 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:41:06.373015 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.372947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:52275)
0809 00:41:06.372985 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.373003 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:41:06.373148 26420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:41:06.375749 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26279
I20260809 00:41:06.376163 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:38441
--local_ip_for_outbound_sockets=127.7.105.67
--tserver_master_addrs=127.7.105.126:38803
--webserver_port=43939
--webserver_interface=127.7.105.67
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:44567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:06.460247 26424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:06.460505 26424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:06.460546 26424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:06.461737 26424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:06.461812 26424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:06.461859 26424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:06.463392 26424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:44567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:38441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=43939
--tserver_master_addrs=127.7.105.126:38803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:06.463676 26424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:06.463929 26424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:06.469316 26430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:06.469326 26429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.469594 26424 server_base.cc:1034] running on GCE node
W20260809 00:41:06.469332 26432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:06.469945 26424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:06.470206 26424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:06.471370 26424 hybrid_clock.cc:648] HybridClock initialized: now 1786236066471360 us; error 30 us; skew 500 ppm
Aug 09 00:41:06 dist-test-slave-1zqn krb5kdc[24853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:06.474032 26424 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:41:06.474532 26424 webserver.cc:460] Webserver started at http://127.7.105.67:43939/ using document root <none> and password file <none>
I20260809 00:41:06.474802 26424 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:06.474876 26424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:06.476373 26424 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:06.477077 26439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.477412 26424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:06.477531 26424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "5c5b9bbae022415aac2ecf41a2b244ae"
format_stamp: "Formatted at 2026-08-09 00:40:24 on dist-test-slave-1zqn"
server_key: "7048ecdd4c444e6e9a735cdac5edbf33"
server_key_iv: "b20921618af457ea33d2bc5820a0bdc8"
server_key_version: "encryptionkey@0"
I20260809 00:41:06.477917 26424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:06.488240 26424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:06.490166 26424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:06.490340 26424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:06.490598 26424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:06.490926 26424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:06.490984 26424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.491027 26424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:06.491052 26424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:06.497694 26424 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:38441
I20260809 00:41:06.497750 26562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:38441 every 8 connection(s)
I20260809 00:41:06.498119 26424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:41:06.499198 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.498811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:54273)
0809 00:41:06.498970 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.499043 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260809 00:41:06.499672 26563 heartbeater.cc:644] Failed to heartbeat to 127.7.105.126:38803 (0 consecutive failures): Network error: Failed to ping master at 127.7.105.126:38803: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
I20260809 00:41:06.499989 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.499897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:44825)
0809 00:41:06.499944 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.499972 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:41:06.500313 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:06.500236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:35343)
0809 00:41:06.500276 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:06.500299 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 00:41:06.500445 26563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:41:06.502110 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26424
I20260809 00:41:07.306733 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.306503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:40883)
0809 00:41:07.306628 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:07.306664 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260809 00:41:07.373754 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.373520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:59397)
0809 00:41:07.373662 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:07.373699 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260809 00:41:07.437114 26570 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:07.437281 26570 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:07.437668 26570 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
Aug 09 00:41:07 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.67@KRBTEST.COM
Aug 09 00:41:07 dist-test-slave-1zqn krb5kdc[24853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236066, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.66@KRBTEST.COM
I20260809 00:41:07.449304 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.437877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:38441 (local address 127.7.105.65:38461)
0809 00:41:07.438216 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:07.438225 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:07.438289 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:07.439181 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:07.439182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:07.439463 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:07.439670 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:07.439676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.440967 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.440968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:07.441786 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:07.441789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.442382 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.442383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:07.442494 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:07.443152 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:07.443157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:07.444934 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:07.447261 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447269 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:07.447271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:07.447689 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:07.447992 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:07.447993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:07.448167 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:07.448677 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:07.448685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:07.448846 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 00:41:07.448983 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 00:41:07.449176 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":55,"thread_start_us":61,"threads_started":1}
I20260809 00:41:07.449321 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.438070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:38461 (local address 127.7.105.67:38441)
0809 00:41:07.438226 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:07.438233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:07.438279 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:41:07.438823 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:07.438826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:07.438894 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:07.439047 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:07.439790 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.440854 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.441877 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.442285 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.442410 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:07.445295 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:07.445299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:07.445299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:07.445309 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:07.447154 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447783 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:07.447786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:07.447787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:07.447914 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:07.448242 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:07.448243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:07.448243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:07.448577 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:07.448669 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:07.449066 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 00:41:07.449218 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 00:41:07.449340 26572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.437991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:58207 (local address 127.7.105.66:39185)
0809 00:41:07.438169 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:07.438184 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:07.438221 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:41:07.438838 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:07.438840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:07.438892 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:07.439047 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:07.439789 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.440788 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.441882 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.442285 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.442410 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:07.445300 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:07.445303 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:07.445304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:07.445314 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:07.447242 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447783 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:07.447786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:07.447787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:07.447914 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:07.448242 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:07.448243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:07.448243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:07.448606 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:07.448682 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:07.449092 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 00:41:07.449233 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260809 00:41:07.449323 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.437837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39185 (local address 127.7.105.65:58207)
0809 00:41:07.437902 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:07.437915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:07.437979 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:07.439110 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:07.439113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:07.439464 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:07.439684 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:07.439686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.440971 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.440973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:07.441785 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:07.441787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:07.442381 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:07.442382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:07.442492 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:07.443144 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:07.443156 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:07.445212 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:07.447308 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:07.447309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:07.447689 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:07.447992 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:07.447993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:07.447994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:07.448167 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:07.448677 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:07.448685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:07.448846 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 00:41:07.448983 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 00:41:07.449231 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":113}
W20260809 00:41:07.450541 26148 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
W20260809 00:41:07.450783 26151 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
I20260809 00:41:07.450852 26151 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37a4fb345a14126bff4534693f0a469; no voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae
I20260809 00:41:07.450971 26570 raft_consensus.cc:2726] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:41:07.500982 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:07.500780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:41695)
0809 00:41:07.500898 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:07.500944 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260809 00:41:08.307538 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:08.307294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:35897)
0809 00:41:08.307447 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:08.307489 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260809 00:41:08.374519 26577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:08.374321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:36463)
0809 00:41:08.374441 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:08.374477 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260809 00:41:08.501726 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:08.501494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:46703)
0809 00:41:08.501653 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:08.501687 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260809 00:41:09.143565 26579 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:09.143664 26579 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:09.143801 26579 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
W20260809 00:41:09.144261 26148 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
W20260809 00:41:09.144354 26151 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
I20260809 00:41:09.144467 26151 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37a4fb345a14126bff4534693f0a469; no voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae
I20260809 00:41:09.144582 26579 raft_consensus.cc:2726] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:41:09.308254 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:09.308032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:37617)
0809 00:41:09.308168 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:09.308204 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260809 00:41:09.375409 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:09.375132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:52319)
0809 00:41:09.375307 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:09.375356 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260809 00:41:09.502630 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:09.502386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:43603)
0809 00:41:09.502528 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:09.502585 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260809 00:41:10.309018 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:10.308769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:58285)
0809 00:41:10.308925 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:10.308967 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260809 00:41:10.376259 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:10.375989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:37725)
0809 00:41:10.376168 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:10.376212 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260809 00:41:10.503327 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:10.503114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:36793)
0809 00:41:10.503239 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:10.503283 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260809 00:41:10.828763 26588 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:10.828871 26588 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:10.829129 26588 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
W20260809 00:41:10.829594 26148 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
W20260809 00:41:10.829653 26151 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
I20260809 00:41:10.829680 26148 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37a4fb345a14126bff4534693f0a469; no voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae
I20260809 00:41:10.829757 26588 raft_consensus.cc:2726] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:41:11.309795 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:11.309545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:58981)
0809 00:41:11.309711 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:11.309746 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260809 00:41:11.377009 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:11.376774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:49029)
0809 00:41:11.376918 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:11.376962 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260809 00:41:11.504238 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:11.503848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:43941)
0809 00:41:11.503997 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:11.504040 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260809 00:41:12.310575 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:12.310338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:48455)
0809 00:41:12.310475 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:12.310518 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260809 00:41:12.377708 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:12.377469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:57063)
0809 00:41:12.377622 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:12.377661 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":62,"threads_started":1}
I20260809 00:41:12.504973 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:12.504731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:48473)
0809 00:41:12.504880 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:12.504923 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260809 00:41:13.311336 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:13.311083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:55969)
0809 00:41:13.311242 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:13.311285 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260809 00:41:13.378459 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:13.378229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:42347)
0809 00:41:13.378377 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:13.378413 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 00:41:13.505713 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:13.505452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:32789)
0809 00:41:13.505625 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:13.505661 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260809 00:41:14.312199 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:14.311915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:37955)
0809 00:41:14.312099 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:14.312148 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260809 00:41:14.379205 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:14.378958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:59287)
0809 00:41:14.379110 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:14.379155 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260809 00:41:14.506414 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:14.506181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:36409)
0809 00:41:14.506320 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:14.506364 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260809 00:41:15.182634 26601 raft_consensus.cc:482] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:15.182753 26601 raft_consensus.cc:504] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5b9bbae022415aac2ecf41a2b244ae" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 38441 } } peers { permanent_uuid: "41460b329b14402c9d422fb6215aeb3b" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39185 } } peers { permanent_uuid: "b37a4fb345a14126bff4534693f0a469" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34425 } }
I20260809 00:41:15.183068 26601 leader_election.cc:288] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441), 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185)
W20260809 00:41:15.183481 26148 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41460b329b14402c9d422fb6215aeb3b (127.7.105.66:39185): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
W20260809 00:41:15.183575 26151 leader_election.cc:341] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c5b9bbae022415aac2ecf41a2b244ae (127.7.105.67:38441): Not found: Tablet not found: c981f737b3594fc09908007389d68e2a
I20260809 00:41:15.183678 26151 leader_election.cc:302] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37a4fb345a14126bff4534693f0a469; no voters: 41460b329b14402c9d422fb6215aeb3b, 5c5b9bbae022415aac2ecf41a2b244ae
I20260809 00:41:15.183745 26601 raft_consensus.cc:2726] T c981f737b3594fc09908007389d68e2a P b37a4fb345a14126bff4534693f0a469 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:41:15.312973 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:15.312743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:51681)
0809 00:41:15.312879 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:15.312923 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260809 00:41:15.380004 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:15.379760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:55713)
0809 00:41:15.379899 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:15.379940 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260809 00:41:15.507284 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:15.506974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:41659)
0809 00:41:15.507168 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:15.507219 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":97,"threads_started":1}
I20260809 00:41:16.313725 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:16.313492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:43835)
0809 00:41:16.313632 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:16.313676 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260809 00:41:16.380782 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:16.380525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:59931)
0809 00:41:16.380695 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:16.380735 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260809 00:41:16.508153 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:16.507845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:59321)
0809 00:41:16.508010 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:16.508087 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260809 00:41:17.314528 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:17.314300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:56427)
0809 00:41:17.314441 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:17.314479 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260809 00:41:17.381659 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:17.381371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:49871)
0809 00:41:17.381564 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:17.381610 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260809 00:41:17.509018 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:17.508718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:60815)
0809 00:41:17.508915 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:17.508962 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":95,"threads_started":1}
I20260809 00:41:18.315290 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:18.315058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:53747)
0809 00:41:18.315200 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:18.315241 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260809 00:41:18.382344 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:18.382135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:50457)
0809 00:41:18.382259 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:18.382296 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260809 00:41:18.509864 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:18.509631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:53125)
0809 00:41:18.509775 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:18.509814 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 00:41:19.316026 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:19.315758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:50001)
0809 00:41:19.315931 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:19.315977 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260809 00:41:19.383127 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:19.382867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:43925)
0809 00:41:19.383035 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:19.383077 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260809 00:41:19.510641 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:19.510389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:35637)
0809 00:41:19.510534 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:19.510574 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260809 00:41:20.317143 26619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:20.316881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:41543)
0809 00:41:20.317045 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:20.317094 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260809 00:41:20.383908 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:20.383687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:50739)
0809 00:41:20.383822 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:20.383855 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 00:41:20.511384 26621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:20.511143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:60501)
0809 00:41:20.511299 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:20.511331 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260809 00:41:21.318086 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:21.317826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.65:51233)
0809 00:41:21.317995 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:21.318032 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260809 00:41:21.384804 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:21.384532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.66:48397)
0809 00:41:21.384707 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:21.384753 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260809 00:41:21.509230 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26097
I20260809 00:41:21.512316 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:21.511955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38803 (local address 127.7.105.67:41697)
0809 00:41:21.512194 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:21.512250 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.126:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":59,"threads_started":1}
I20260809 00:41:21.516777 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26279
I20260809 00:41:21.523952 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26424
2026-08-09T00:41:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58579 ms)
[----------] 30 tests from SecurityITest (238494 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:21 dist-test-slave-1zqn krb5kdc[26627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:21 dist-test-slave-1zqn krb5kdc[26627](info): set up 2 sockets
Aug 09 00:41:21 dist-test-slave-1zqn krb5kdc[26627](info): commencing operation
krb5kdc: starting...
W20260809 00:41:22.731424 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:22 dist-test-slave-1zqn krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:22.863564 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41129
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:22.947854 26641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:22.948128 26641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:22.948179 26641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:22.949318 26641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:22.949388 26641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:22.949412 26641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:22.949435 26641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:22.949465 26641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:22.950937 26641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:22.951228 26641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:22.951458 26641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:22.957168 26648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:22.957172 26647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:22.957183 26641 server_base.cc:1034] running on GCE node
W20260809 00:41:22.957172 26650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:22.957764 26641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:22.958024 26641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:22.959213 26641 hybrid_clock.cc:648] HybridClock initialized: now 1786236082959197 us; error 32 us; skew 500 ppm
Aug 09 00:41:22 dist-test-slave-1zqn krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236082, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:22.962020 26641 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:22.962451 26641 webserver.cc:460] Webserver started at http://127.7.105.126:34401/ using document root <none> and password file <none>
I20260809 00:41:22.962699 26641 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:22.962769 26641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:22.962918 26641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:22.964025 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "0d1579f1a07b4170984a119a8f9a63d1"
format_stamp: "Formatted at 2026-08-09 00:41:22 on dist-test-slave-1zqn"
server_key: "ccfcd173584077a930bfdd5134eec846"
server_key_iv: "7a3a33321199edfa625ca8de5d606915"
server_key_version: "encryptionkey@0"
I20260809 00:41:22.964460 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "0d1579f1a07b4170984a119a8f9a63d1"
format_stamp: "Formatted at 2026-08-09 00:41:22 on dist-test-slave-1zqn"
server_key: "ccfcd173584077a930bfdd5134eec846"
server_key_iv: "7a3a33321199edfa625ca8de5d606915"
server_key_version: "encryptionkey@0"
I20260809 00:41:22.965950 26641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:41:22.966739 26658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:22.966912 26641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:41:22.967063 26641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "0d1579f1a07b4170984a119a8f9a63d1"
format_stamp: "Formatted at 2026-08-09 00:41:22 on dist-test-slave-1zqn"
server_key: "ccfcd173584077a930bfdd5134eec846"
server_key_iv: "7a3a33321199edfa625ca8de5d606915"
server_key_version: "encryptionkey@0"
I20260809 00:41:22.967141 26641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:22.987603 26641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:22.989791 26641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:22.990020 26641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:22.994197 26641 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:41129
I20260809 00:41:22.994236 26716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:41129 every 8 connection(s)
I20260809 00:41:22.994638 26641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:22.995627 26718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:22.998045 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: Bootstrap starting.
I20260809 00:41:22.998656 26718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:22.998906 26718 log.cc:824] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:22.999432 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26641
I20260809 00:41:22.999565 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:22.999719 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: No bootstrap required, opened a new log
I20260809 00:41:22.999820 7589 external_mini_cluster.cc:1428] Setting key e6d6fb59726a5d831a95f77b1ec4e26c
I20260809 00:41:23.001796 26718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } }
I20260809 00:41:23.001977 26718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:23.002074 26718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1579f1a07b4170984a119a8f9a63d1, State: Initialized, Role: FOLLOWER
I20260809 00:41:23.002242 26718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } }
I20260809 00:41:23.002339 26718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:23.002378 26718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:23.002432 26718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:23.003508 26718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } }
I20260809 00:41:23.003649 26718 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1579f1a07b4170984a119a8f9a63d1; no voters:
I20260809 00:41:23.003854 26718 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:23.003878 26724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:23.003994 26724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [term 1 LEADER]: Becoming Leader. State: Replica: 0d1579f1a07b4170984a119a8f9a63d1, State: Running, Role: LEADER
I20260809 00:41:23.004166 26718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:23.004168 26724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } }
I20260809 00:41:23.004840 26725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d1579f1a07b4170984a119a8f9a63d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } } }
I20260809 00:41:23.004865 26726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1579f1a07b4170984a119a8f9a63d1. Latest consensus state: current_term: 1 leader_uuid: "0d1579f1a07b4170984a119a8f9a63d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1579f1a07b4170984a119a8f9a63d1" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41129 } } }
I20260809 00:41:23.004943 26725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:23.004966 26726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:23.005162 26734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:23.005837 26734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:23.007966 26734 catalog_manager.cc:1347] Generated new cluster ID: dca4db1c2663407fbd41488dae7420cc
I20260809 00:41:23.008026 26734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:23.013773 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.000558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.7.105.126:41129)
0809 00:41:23.000766 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.000774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.000804 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:41:23.001781 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.001790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.001851 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.001987 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.002896 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.004186 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.005423 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.006236 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.006351 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.009659 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.009663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.009663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:23.009677 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.011902 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.012376 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.012379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.012379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.012519 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.012774 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.012775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.012775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.013112 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.013272 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.013383 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.013530 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260809 00:41:23.023239 26734 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:23.023797 26734 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:23.032191 26734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: Generated new TSK 0
I20260809 00:41:23.032408 26734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:41:23.063292 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:23.149204 26748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:23.149412 26748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:23.149444 26748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:23.150532 26748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:23.150573 26748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:23.150599 26748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:23.152004 26748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:23.152225 26748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:23.152424 26748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:23.158070 26754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.158053 26753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.158200 26756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.158218 26748 server_base.cc:1034] running on GCE node
I20260809 00:41:23.158511 26748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:23.158792 26748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:23.159940 26748 hybrid_clock.cc:648] HybridClock initialized: now 1786236083159925 us; error 32 us; skew 500 ppm
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:23.162712 26748 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:41:23.163159 26748 webserver.cc:460] Webserver started at http://127.7.105.65:37997/ using document root <none> and password file <none>
I20260809 00:41:23.163422 26748 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:23.163496 26748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:23.163651 26748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:23.164786 26748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "c2feac515f024ef7a599f347ef5e4a44"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "38c46d3765c088e86a0ccc5e251efb1b"
server_key_iv: "732c19bde4c70406834e67e51eaade8f"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.165158 26748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "c2feac515f024ef7a599f347ef5e4a44"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "38c46d3765c088e86a0ccc5e251efb1b"
server_key_iv: "732c19bde4c70406834e67e51eaade8f"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.166592 26748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:41:23.167495 26763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.167724 26748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:23.167840 26748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "c2feac515f024ef7a599f347ef5e4a44"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "38c46d3765c088e86a0ccc5e251efb1b"
server_key_iv: "732c19bde4c70406834e67e51eaade8f"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.167917 26748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:23.191329 26748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:23.193986 26748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:23.194186 26748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:23.194463 26748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:23.194811 26748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:23.194867 26748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.194909 26748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:23.194938 26748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.201853 26748 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:41293
I20260809 00:41:23.201937 26888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:41293 every 8 connection(s)
I20260809 00:41:23.202301 26748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:23.209496 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26748
I20260809 00:41:23.209605 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:23.209810 7589 external_mini_cluster.cc:1428] Setting key 12ee471d4feaa2c24026e6740f34d131
I20260809 00:41:23.212927 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.203314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:37281 (local address 127.7.105.126:41129)
0809 00:41:23.203416 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.203418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.203608 (+ 190us) server_negotiation.cc:407] Connection header received
0809 00:41:23.204241 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.204242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.204256 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.204313 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.205452 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.205997 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.207062 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.207299 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.207389 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.209758 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.209760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.209760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:23.209768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.211374 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.211822 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.211823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.211824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.211893 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.212232 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.212232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.212233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.212360 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.212439 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.212738 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.212836 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:41:23.213129 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.203265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41129 (local address 127.7.105.65:37281)
0809 00:41:23.203466 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.203517 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.204132 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.204444 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.204455 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.205050 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:23.205323 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.205329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.206125 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.206126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.206964 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.206966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.207403 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.207404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.207549 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.208147 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:23.208156 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:23.209650 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:23.211488 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.211496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.211499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.211739 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.211961 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.211962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.211962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.212140 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.212429 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:23.212437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:23.212586 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.212699 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.212867 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260809 00:41:23.213636 26889 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41129
I20260809 00:41:23.213757 26889 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:23.213940 26889 heartbeater.cc:511] Master 127.7.105.126:41129 requested a full tablet report, sending...
I20260809 00:41:23.214461 26678 ts_manager.cc:194] Registered new tserver with Master: c2feac515f024ef7a599f347ef5e4a44 (127.7.105.65:41293)
I20260809 00:41:23.215191 26678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:37281
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:41:23.237280 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:23.324404 26895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:23.324605 26895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:23.324630 26895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:23.325703 26895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:23.325737 26895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:23.325762 26895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:23.327175 26895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:23.327464 26895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:23.327715 26895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:23.332979 26905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.333076 26902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.333003 26903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.333264 26895 server_base.cc:1034] running on GCE node
I20260809 00:41:23.333464 26895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:23.333685 26895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:23.334861 26895 hybrid_clock.cc:648] HybridClock initialized: now 1786236083334849 us; error 30 us; skew 500 ppm
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:23.337487 26895 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:41:23.337970 26895 webserver.cc:460] Webserver started at http://127.7.105.66:34143/ using document root <none> and password file <none>
I20260809 00:41:23.338233 26895 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:23.338304 26895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:23.338459 26895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:23.339531 26895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "524cd34722b94fb9a996f3d26826126d"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "2074da0b1b9db0c665554a27833312e5"
server_key_iv: "511f39e96c19979a548f4ec52e414507"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.339903 26895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "524cd34722b94fb9a996f3d26826126d"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "2074da0b1b9db0c665554a27833312e5"
server_key_iv: "511f39e96c19979a548f4ec52e414507"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.341370 26895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:23.342286 26913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.342501 26895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:23.342625 26895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "524cd34722b94fb9a996f3d26826126d"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "2074da0b1b9db0c665554a27833312e5"
server_key_iv: "511f39e96c19979a548f4ec52e414507"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.342708 26895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:23.365837 26895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:23.367972 26895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:23.368197 26895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:23.368462 26895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:23.368811 26895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:23.368866 26895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.368914 26895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:23.368943 26895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.375293 26895 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:41865
I20260809 00:41:23.375416 27032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:41865 every 8 connection(s)
I20260809 00:41:23.375730 26895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:23.383239 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 26895
I20260809 00:41:23.383368 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:23.383579 7589 external_mini_cluster.cc:1428] Setting key 0a5ef02131b79aec4f7f600da91938cf
I20260809 00:41:23.386490 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.376590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:58243 (local address 127.7.105.126:41129)
0809 00:41:23.376702 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.376705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.376906 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:41:23.377382 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.377383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.377397 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.377480 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.378459 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.379176 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.380232 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.380453 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.380533 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.382984 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.382986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.382987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:23.382996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.384790 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.385329 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.385330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.385330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.385390 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.385691 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.385691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.385692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.385804 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.385860 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.386194 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.386316 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 00:41:23.386677 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.376557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41129 (local address 127.7.105.66:58243)
0809 00:41:23.376766 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.376804 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.377290 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.377535 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.377538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.378137 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:23.378335 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.378340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.379279 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.379280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.380122 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.380125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.380546 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.380547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.380667 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.381255 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:23.381263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:23.382839 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:23.384894 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.384902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.384905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.385246 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.385462 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.385463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.385464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.385622 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.385872 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:23.385879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:23.386052 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.386168 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.386364 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
I20260809 00:41:23.387200 27033 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41129
I20260809 00:41:23.387331 27033 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:23.387559 27033 heartbeater.cc:511] Master 127.7.105.126:41129 requested a full tablet report, sending...
I20260809 00:41:23.387985 26678 ts_manager.cc:194] Registered new tserver with Master: 524cd34722b94fb9a996f3d26826126d (127.7.105.66:41865)
I20260809 00:41:23.388608 26678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:58243
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:41:23.415342 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:23.500488 27039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:23.500752 27039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:23.500797 27039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:23.502094 27039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:23.502171 27039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:23.502218 27039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:23.503772 27039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:41129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:23.504058 27039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:23.504351 27039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:23.509728 27047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.509843 27039 server_base.cc:1034] running on GCE node
W20260809 00:41:23.509749 27050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.509781 27046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.510251 27039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:23.510454 27039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:23.511632 27039 hybrid_clock.cc:648] HybridClock initialized: now 1786236083511606 us; error 37 us; skew 500 ppm
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:23.514274 27039 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:41:23.514734 27039 webserver.cc:460] Webserver started at http://127.7.105.67:37575/ using document root <none> and password file <none>
I20260809 00:41:23.514921 27039 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:23.514961 27039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:23.515060 27039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:23.516142 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "1a42e4f2edca44b18a9fad04161cb2d8"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "fd4406fa865d0be4edb91b42027048a2"
server_key_iv: "92702b1e60dcd94abca2c68a0719d3d6"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.516508 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "1a42e4f2edca44b18a9fad04161cb2d8"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "fd4406fa865d0be4edb91b42027048a2"
server_key_iv: "92702b1e60dcd94abca2c68a0719d3d6"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.517827 27039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:23.518617 27057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.518806 27039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:23.518918 27039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "1a42e4f2edca44b18a9fad04161cb2d8"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "fd4406fa865d0be4edb91b42027048a2"
server_key_iv: "92702b1e60dcd94abca2c68a0719d3d6"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.519039 27039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:23.528460 27039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:23.530750 27039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:23.530913 27039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:23.531157 27039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:23.531486 27039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:23.531543 27039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.531584 27039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:23.531629 27039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.537861 27039 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:34319
I20260809 00:41:23.537907 27182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:34319 every 8 connection(s)
I20260809 00:41:23.538231 27039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:41:23.541179 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27039
I20260809 00:41:23.541319 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:23.541560 7589 external_mini_cluster.cc:1428] Setting key d76e2cd0ac7721cec7933168285a6288
Aug 09 00:41:23 dist-test-slave-1zqn krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:23.548309 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.539026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:43393 (local address 127.7.105.126:41129)
0809 00:41:23.539099 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.539101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.539312 (+ 211us) server_negotiation.cc:407] Connection header received
0809 00:41:23.539783 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.539784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.539794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.539853 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.540855 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.541405 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.542610 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.542794 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.542845 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.545231 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.545232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.545232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:23.545240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.546900 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.547378 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.547379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.547379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.547457 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.547688 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.547689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.547689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.547802 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.547857 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.548158 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.548239 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:41:23.548548 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.538999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41129 (local address 127.7.105.67:43393)
0809 00:41:23.539175 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.539217 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.539702 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.539906 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.539909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.540577 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:23.540774 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.540779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.541512 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.541513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.542538 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.542540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.542864 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.542864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.543027 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.543485 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:23.543493 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:23.545139 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:23.547054 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.547061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.547063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.547304 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.547531 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:23.547532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:23.547532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:23.547629 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:23.547860 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:23.547864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:23.547976 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.548156 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.548287 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260809 00:41:23.549052 27183 heartbeater.cc:347] Connected to a master server at 127.7.105.126:41129
I20260809 00:41:23.549149 27183 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:23.549297 27183 heartbeater.cc:511] Master 127.7.105.126:41129 requested a full tablet report, sending...
I20260809 00:41:23.549634 26678 ts_manager.cc:194] Registered new tserver with Master: 1a42e4f2edca44b18a9fad04161cb2d8 (127.7.105.67:34319)
I20260809 00:41:23.549971 26678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:43393
I20260809 00:41:23.553364 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:23.560606 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.554734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.7.105.126:41129)
0809 00:41:23.554785 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.554787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.554930 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:41:23.554965 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.554965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.554974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.555023 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.555552 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.555998 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.556731 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.556906 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.556982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.558109 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.558110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.558110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:23.558116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.559496 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.559802 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.559802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.559802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.559852 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:23.560025 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:23.560026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:23.560026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:23.560244 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.560323 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.560418 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.560476 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:23.562431 26678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:23.564118 26678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:23.571123 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.566988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.7.105.65:41293)
0809 00:41:23.567203 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.567208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.567220 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:41:23.567260 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.567261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.567274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.567469 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:23.567578 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.568502 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.569888 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.570709 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.570781 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.570844 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.570988 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.571040 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":143,"threads_started":1}
I20260809 00:41:23.571115 27197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.566698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41293 (local address 127.0.0.1:37206)
0809 00:41:23.566860 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.566904 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.566973 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.567407 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.567411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.567413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:23.567504 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.567506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.568599 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.568601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.569797 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.569800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.570783 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.570784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.570866 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.570880 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.570972 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.571044 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260809 00:41:23.571295 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.566928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:34319 (local address 127.0.0.1:58762)
0809 00:41:23.567344 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.567376 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.567433 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.567694 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.567695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.567697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:23.567738 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.567740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.569048 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.569049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.570089 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.570091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.571050 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.571051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.571117 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.571121 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.571154 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.571195 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":73,"threads_started":1}
I20260809 00:41:23.571636 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.567197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.7.105.67:34319)
0809 00:41:23.567384 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.567388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.567426 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:41:23.567538 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.567541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.567558 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.567642 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:23.567816 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.568933 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.570168 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.570921 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.571428 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.571492 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.571515 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.571567 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260809 00:41:23.572091 26819 tablet_service.cc:1467] Processing CreateTablet for tablet e6f8ea8678604843a0acdd78b061a2fb (DEFAULT_TABLE table=test-table [id=4fe4c7c7cd3c4870b8ce2e6ccec99e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:23.572315 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.566706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:41865 (local address 127.0.0.1:40554)
0809 00:41:23.567029 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.567040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.567095 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.567951 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.567953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.567955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:23.568005 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.568006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.569257 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.569258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.570437 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.570441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.572073 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.572075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.572135 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.572139 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.572196 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.572235 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":92,"threads_started":1}
I20260809 00:41:23.572356 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.566849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.7.105.66:41865)
0809 00:41:23.567787 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.567792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.567801 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:23.567838 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.567841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.567858 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.567936 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:23.568113 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.569152 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.570519 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.571897 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.571964 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.572015 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.572226 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.572280 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":90,"threads_started":1}
I20260809 00:41:23.572279 27109 tablet_service.cc:1467] Processing CreateTablet for tablet e6f8ea8678604843a0acdd78b061a2fb (DEFAULT_TABLE table=test-table [id=4fe4c7c7cd3c4870b8ce2e6ccec99e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:23.572504 26819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f8ea8678604843a0acdd78b061a2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:23.572670 27109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f8ea8678604843a0acdd78b061a2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:23.572989 26963 tablet_service.cc:1467] Processing CreateTablet for tablet e6f8ea8678604843a0acdd78b061a2fb (DEFAULT_TABLE table=test-table [id=4fe4c7c7cd3c4870b8ce2e6ccec99e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:23.573346 26963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f8ea8678604843a0acdd78b061a2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:23.575182 27202 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: Bootstrap starting.
I20260809 00:41:23.575898 27202 tablet_bootstrap.cc:654] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:23.576140 27203 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Bootstrap starting.
I20260809 00:41:23.576598 27202 log.cc:824] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:23.576627 27204 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Bootstrap starting.
I20260809 00:41:23.577003 27203 tablet_bootstrap.cc:654] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:23.577275 27204 tablet_bootstrap.cc:654] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:23.577456 27202 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: No bootstrap required, opened a new log
I20260809 00:41:23.577540 27202 ts_tablet_manager.cc:1397] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:23.577605 27203 log.cc:824] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:23.577895 27204 log.cc:824] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:23.578425 27203 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: No bootstrap required, opened a new log
I20260809 00:41:23.578512 27203 ts_tablet_manager.cc:1397] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:23.579167 27202 raft_consensus.cc:348] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.579312 27202 raft_consensus.cc:374] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:23.579335 27202 raft_consensus.cc:729] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2feac515f024ef7a599f347ef5e4a44, State: Initialized, Role: FOLLOWER
I20260809 00:41:23.579475 27202 consensus_queue.cc:260] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.579787 27202 ts_tablet_manager.cc:1428] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:41:23.579818 26889 heartbeater.cc:503] Master 127.7.105.126:41129 was elected leader, sending a full tablet report...
I20260809 00:41:23.580418 27204 tablet_bootstrap.cc:492] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: No bootstrap required, opened a new log
I20260809 00:41:23.580353 27203 raft_consensus.cc:348] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.580489 27204 ts_tablet_manager.cc:1397] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:41:23.580504 27203 raft_consensus.cc:374] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:23.580550 27203 raft_consensus.cc:729] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a42e4f2edca44b18a9fad04161cb2d8, State: Initialized, Role: FOLLOWER
I20260809 00:41:23.580660 27203 consensus_queue.cc:260] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.580933 27203 ts_tablet_manager.cc:1428] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:41:23.580940 27183 heartbeater.cc:503] Master 127.7.105.126:41129 was elected leader, sending a full tablet report...
I20260809 00:41:23.582464 27204 raft_consensus.cc:348] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.582612 27204 raft_consensus.cc:374] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:23.582656 27204 raft_consensus.cc:729] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524cd34722b94fb9a996f3d26826126d, State: Initialized, Role: FOLLOWER
I20260809 00:41:23.582763 27204 consensus_queue.cc:260] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.583033 27204 ts_tablet_manager.cc:1428] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:41:23.583143 27033 heartbeater.cc:503] Master 127.7.105.126:41129 was elected leader, sending a full tablet report...
I20260809 00:41:23.624846 27208 raft_consensus.cc:482] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:23.624979 27208 raft_consensus.cc:504] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.625381 27208 leader_election.cc:288] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 524cd34722b94fb9a996f3d26826126d (127.7.105.66:41865), 1a42e4f2edca44b18a9fad04161cb2d8 (127.7.105.67:34319)
W20260809 00:41:23.626389 27035 tablet.cc:2369] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:23.628724 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.625713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:60719 (local address 127.7.105.66:41865)
0809 00:41:23.625782 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.625784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.625793 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:23.625823 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.625824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.625836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.625901 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:23.626108 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.626796 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.627790 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.628348 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.628441 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.628452 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.628622 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.628660 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:41:23.628726 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.625611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:41865 (local address 127.7.105.65:60719)
0809 00:41:23.625676 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.625690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.625736 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.625910 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.625912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.625914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:23.625999 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.626001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.626877 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.626879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.627698 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.627700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.628443 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.628452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.628538 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.628542 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.628603 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.628650 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:41:23.628804 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.625820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:46875 (local address 127.7.105.67:34319)
0809 00:41:23.625892 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.625894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.626056 (+ 162us) server_negotiation.cc:407] Connection header received
0809 00:41:23.626142 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.626143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.626160 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.626203 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:23.626402 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.627012 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.628015 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.628412 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.628503 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.628512 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.628690 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.628730 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:41:23.628832 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.625742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:34319 (local address 127.7.105.65:46875)
0809 00:41:23.625967 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.625978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:23.626041 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:23.626228 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:23.626229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:23.626231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:23.626319 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.626322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.627098 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.627099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.627896 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:23.627898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.628520 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.628521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:23.628601 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.628606 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:41:23.628670 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:41:23.628713 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":162,"threads_started":1}
I20260809 00:41:23.629504 27130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f8ea8678604843a0acdd78b061a2fb" candidate_uuid: "c2feac515f024ef7a599f347ef5e4a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" is_pre_election: true
I20260809 00:41:23.629539 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f8ea8678604843a0acdd78b061a2fb" candidate_uuid: "c2feac515f024ef7a599f347ef5e4a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524cd34722b94fb9a996f3d26826126d" is_pre_election: true
I20260809 00:41:23.629680 27130 raft_consensus.cc:2454] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2feac515f024ef7a599f347ef5e4a44 in term 0.
I20260809 00:41:23.629699 26984 raft_consensus.cc:2454] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2feac515f024ef7a599f347ef5e4a44 in term 0.
I20260809 00:41:23.629907 26767 leader_election.cc:302] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a42e4f2edca44b18a9fad04161cb2d8, c2feac515f024ef7a599f347ef5e4a44; no voters:
I20260809 00:41:23.630049 27208 raft_consensus.cc:2781] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:23.630110 27208 raft_consensus.cc:482] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:23.630136 27208 raft_consensus.cc:3037] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:23.630856 27208 raft_consensus.cc:504] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.630991 27208 leader_election.cc:288] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [CANDIDATE]: Term 1 election: Requested vote from peers 524cd34722b94fb9a996f3d26826126d (127.7.105.66:41865), 1a42e4f2edca44b18a9fad04161cb2d8 (127.7.105.67:34319)
I20260809 00:41:23.631204 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f8ea8678604843a0acdd78b061a2fb" candidate_uuid: "c2feac515f024ef7a599f347ef5e4a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524cd34722b94fb9a996f3d26826126d"
I20260809 00:41:23.631204 27130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f8ea8678604843a0acdd78b061a2fb" candidate_uuid: "c2feac515f024ef7a599f347ef5e4a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a42e4f2edca44b18a9fad04161cb2d8"
I20260809 00:41:23.631294 26984 raft_consensus.cc:3037] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:23.631291 27130 raft_consensus.cc:3037] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:23.632301 27130 raft_consensus.cc:2454] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2feac515f024ef7a599f347ef5e4a44 in term 1.
I20260809 00:41:23.632378 26984 raft_consensus.cc:2454] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2feac515f024ef7a599f347ef5e4a44 in term 1.
I20260809 00:41:23.632527 26767 leader_election.cc:302] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524cd34722b94fb9a996f3d26826126d, c2feac515f024ef7a599f347ef5e4a44; no voters:
I20260809 00:41:23.632683 27208 raft_consensus.cc:2781] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:23.632899 27208 raft_consensus.cc:686] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [term 1 LEADER]: Becoming Leader. State: Replica: c2feac515f024ef7a599f347ef5e4a44, State: Running, Role: LEADER
I20260809 00:41:23.633025 27208 consensus_queue.cc:237] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } }
I20260809 00:41:23.633893 26678 catalog_manager.cc:5495] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2feac515f024ef7a599f347ef5e4a44 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "c2feac515f024ef7a599f347ef5e4a44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2feac515f024ef7a599f347ef5e4a44" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 41293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 } health_report { overall_health: UNKNOWN } } }
I20260809 00:41:23.643664 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.641447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.7.105.65:41293)
0809 00:41:23.641515 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.641517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.641525 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:23.641556 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.641557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.641575 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.641630 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:23.641754 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.642259 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.643013 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.643189 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.643240 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.643379 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:23.643479 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:23.643539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.643570 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.643587 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:41:23.645651 26984 raft_consensus.cc:1261] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 1 FOLLOWER]: Refusing update from remote peer c2feac515f024ef7a599f347ef5e4a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:23.646063 27130 raft_consensus.cc:1261] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 1 FOLLOWER]: Refusing update from remote peer c2feac515f024ef7a599f347ef5e4a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:23.646081 27212 consensus_queue.cc:1035] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524cd34722b94fb9a996f3d26826126d" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:23.646340 27212 consensus_queue.cc:1035] T e6f8ea8678604843a0acdd78b061a2fb P c2feac515f024ef7a599f347ef5e4a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a42e4f2edca44b18a9fad04161cb2d8" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:23.648636 27215 mvcc.cc:205] Tried to move back new op lower bound from 7316422998609698816 to 7316422998561488896. Current Snapshot: MvccSnapshot[applied={T|T < 7316422998609698816}]
I20260809 00:41:23.651563 26677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:23.651654 26677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:23.652465 26677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d1579f1a07b4170984a119a8f9a63d1: Sending DeleteTablet for 3 replicas of tablet e6f8ea8678604843a0acdd78b061a2fb
I20260809 00:41:23.652865 26963 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f8ea8678604843a0acdd78b061a2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:23 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:40554
I20260809 00:41:23.652902 27109 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f8ea8678604843a0acdd78b061a2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:23 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:58762
I20260809 00:41:23.653013 26819 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f8ea8678604843a0acdd78b061a2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:23 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:37206
I20260809 00:41:23.653167 27224 tablet_replica.cc:330] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: stopping tablet replica
I20260809 00:41:23.653276 27224 raft_consensus.cc:2229] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:23.653468 27224 raft_consensus.cc:2258] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:23.653496 27225 tablet_replica.cc:330] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: stopping tablet replica
I20260809 00:41:23.653513 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26748
I20260809 00:41:23.653602 27225 raft_consensus.cc:2229] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:23.653702 27225 raft_consensus.cc:2258] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:23.656433 27225 ts_tablet_manager.cc:1905] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:23.656924 27224 ts_tablet_manager.cc:1905] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:23.658610 27224 ts_tablet_manager.cc:1918] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:23.658668 27225 ts_tablet_manager.cc:1918] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:23.658725 27224 log.cc:1196] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/e6f8ea8678604843a0acdd78b061a2fb
I20260809 00:41:23.658740 27225 log.cc:1196] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/e6f8ea8678604843a0acdd78b061a2fb
I20260809 00:41:23.659017 27224 ts_tablet_manager.cc:1939] T e6f8ea8678604843a0acdd78b061a2fb P 524cd34722b94fb9a996f3d26826126d: Deleting consensus metadata
I20260809 00:41:23.659034 27225 ts_tablet_manager.cc:1939] T e6f8ea8678604843a0acdd78b061a2fb P 1a42e4f2edca44b18a9fad04161cb2d8: Deleting consensus metadata
I20260809 00:41:23.659485 26663 catalog_manager.cc:4841] TS 524cd34722b94fb9a996f3d26826126d (127.7.105.66:41865): tablet e6f8ea8678604843a0acdd78b061a2fb (table test-table [id=4fe4c7c7cd3c4870b8ce2e6ccec99e64]) successfully deleted
I20260809 00:41:23.659628 26663 catalog_manager.cc:4841] TS 1a42e4f2edca44b18a9fad04161cb2d8 (127.7.105.67:34319): tablet e6f8ea8678604843a0acdd78b061a2fb (table test-table [id=4fe4c7c7cd3c4870b8ce2e6ccec99e64]) successfully deleted
W20260809 00:41:23.662595 26660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:41293 (error 108)
I20260809 00:41:23.662875 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26895
I20260809 00:41:23.664858 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.663025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:41293 (local address 127.0.0.1:37224)
0809 00:41:23.664784 (+ 1759us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:23.664827 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1729}
W20260809 00:41:23.664991 26660 catalog_manager.cc:4584] TS c2feac515f024ef7a599f347ef5e4a44 (127.7.105.65:41293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6f8ea8678604843a0acdd78b061a2fb: Network error: Client connection negotiation failed: client connection to 127.7.105.65:41293: connect: Connection refused (error 111)
I20260809 00:41:23.670070 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27039
I20260809 00:41:23.678225 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 26641
2026-08-09T00:41:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T00:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:23Z Disabled control of system clock
I20260809 00:41:23.711213 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34015
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:34015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:23.798770 27233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:23.799011 27233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:23.799033 27233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:23.800187 27233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:23.800249 27233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:23.800271 27233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:23.800284 27233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:23.800297 27233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:23.801656 27233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:34015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:23.801918 27233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:23.802253 27233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:23.807583 27239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.807575 27238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.807575 27241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.808125 27233 server_base.cc:1034] running on GCE node
I20260809 00:41:23.808430 27233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:23.808725 27233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:23.809899 27233 hybrid_clock.cc:648] HybridClock initialized: now 1786236083809876 us; error 47 us; skew 500 ppm
I20260809 00:41:23.811316 27233 webserver.cc:460] Webserver started at http://127.7.105.126:35825/ using document root <none> and password file <none>
I20260809 00:41:23.811641 27233 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:23.811733 27233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:23.811877 27233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:23.813033 27233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "01cfdfbfc4374105b4bee7c582e31c26"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "e5fbf3f5f9b5669f77cd981c80cb4b63"
server_key_iv: "bbed38e77bd200e80d98edad90c4a4d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.813345 27233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "01cfdfbfc4374105b4bee7c582e31c26"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "e5fbf3f5f9b5669f77cd981c80cb4b63"
server_key_iv: "bbed38e77bd200e80d98edad90c4a4d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.814778 27233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:41:23.815572 27247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.815780 27233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:23.815901 27233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "01cfdfbfc4374105b4bee7c582e31c26"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "e5fbf3f5f9b5669f77cd981c80cb4b63"
server_key_iv: "bbed38e77bd200e80d98edad90c4a4d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.816109 27233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:23.836931 27233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:23.837255 27233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:23.837368 27233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:23.841573 27233 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:34015
I20260809 00:41:23.841639 27304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:34015 every 8 connection(s)
I20260809 00:41:23.842039 27233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:23.843161 27305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:23.845620 27305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: Bootstrap starting.
I20260809 00:41:23.846261 27305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:23.846529 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27233
I20260809 00:41:23.846613 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:23.846596 27305 log.cc:824] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:23.846815 7589 external_mini_cluster.cc:1428] Setting key cfd1d9dfd39f4cb55de7b236aae16149
I20260809 00:41:23.847393 27305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: No bootstrap required, opened a new log
I20260809 00:41:23.850404 27305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } }
I20260809 00:41:23.850587 27305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:23.850627 27305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01cfdfbfc4374105b4bee7c582e31c26, State: Initialized, Role: FOLLOWER
I20260809 00:41:23.850751 27305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } }
I20260809 00:41:23.850845 27305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:23.850888 27305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:23.850963 27305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:23.851600 27305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } }
I20260809 00:41:23.851718 27305 leader_election.cc:302] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01cfdfbfc4374105b4bee7c582e31c26; no voters:
I20260809 00:41:23.851888 27305 leader_election.cc:288] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:23.851963 27310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:23.852138 27310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [term 1 LEADER]: Becoming Leader. State: Replica: 01cfdfbfc4374105b4bee7c582e31c26, State: Running, Role: LEADER
I20260809 00:41:23.852181 27305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:23.852242 27310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } }
I20260809 00:41:23.852602 27310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cfdfbfc4374105b4bee7c582e31c26. Latest consensus state: current_term: 1 leader_uuid: "01cfdfbfc4374105b4bee7c582e31c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } } }
I20260809 00:41:23.852612 27311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01cfdfbfc4374105b4bee7c582e31c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cfdfbfc4374105b4bee7c582e31c26" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34015 } } }
I20260809 00:41:23.852696 27310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:23.852813 27311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26 [sys.catalog]: This master's current role is: LEADER
W20260809 00:41:23.854059 27326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:23.854190 27326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:23.854280 27320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:23.854350 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:23.847426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.7.105.126:34015)
0809 00:41:23.847653 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:23.847662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:23.847697 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:41:23.848771 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:23.848783 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:23.848867 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:23.849024 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:23.849248 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.850741 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.851615 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:23.852288 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:23.852831 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:23.853257 (+ 426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:23.853262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:23.853262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:23.853510 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:23.853559 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:23.853577 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:23.853930 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:23.853962 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 00:41:23.854051 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260809 00:41:23.854552 27320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:23.856348 27320 catalog_manager.cc:1347] Generated new cluster ID: ea3660baa84046988190e5a69e681f95
I20260809 00:41:23.856407 27320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:23.876880 27320 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:23.877615 27320 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:23.883646 27320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: Generated new TSK 0
I20260809 00:41:23.883869 27320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:23.894651 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--builtin_ntp_servers=127.7.105.84:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:23.981848 27330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:23.982187 27330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:23.982234 27330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:23.983476 27330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:23.983543 27330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:23.983592 27330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:23.985129 27330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:23.985404 27330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:23.985728 27330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:23.991492 27339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.991532 27337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:23.991499 27336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:23.991788 27330 server_base.cc:1034] running on GCE node
I20260809 00:41:23.992019 27330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:23.992298 27330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:23.993451 27330 hybrid_clock.cc:648] HybridClock initialized: now 1786236083993434 us; error 30 us; skew 500 ppm
I20260809 00:41:23.994800 27330 webserver.cc:460] Webserver started at http://127.7.105.65:35703/ using document root <none> and password file <none>
I20260809 00:41:23.995057 27330 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:23.995159 27330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:23.995321 27330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:23.996519 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "ae38ea9dedf745cf805ac4ad70634a92"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "abea6becc640fff8c6da3da7e4a717f4"
server_key_iv: "7293d489ba8e987331a8e599866ca637"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.996901 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "ae38ea9dedf745cf805ac4ad70634a92"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "abea6becc640fff8c6da3da7e4a717f4"
server_key_iv: "7293d489ba8e987331a8e599866ca637"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.998327 27330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:41:23.999229 27345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:23.999468 27330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:23.999586 27330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "ae38ea9dedf745cf805ac4ad70634a92"
format_stamp: "Formatted at 2026-08-09 00:41:23 on dist-test-slave-1zqn"
server_key: "abea6becc640fff8c6da3da7e4a717f4"
server_key_iv: "7293d489ba8e987331a8e599866ca637"
server_key_version: "encryptionkey@0"
I20260809 00:41:23.999678 27330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:24.032508 27330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:24.032886 27330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:24.033051 27330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:24.033327 27330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:24.033726 27330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:24.033785 27330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.033823 27330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:24.033871 27330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.040095 27330 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:43849
I20260809 00:41:24.040197 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:43849 every 8 connection(s)
I20260809 00:41:24.040560 27330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:24.041698 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27330
I20260809 00:41:24.041798 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:24.042006 7589 external_mini_cluster.cc:1428] Setting key 81c041c6ec6ad5d2ecf0178dce8d3dde
I20260809 00:41:24.043143 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--builtin_ntp_servers=127.7.105.84:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:41:24.046928 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.041451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:33857 (local address 127.7.105.126:34015)
0809 00:41:24.041557 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.041560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.041753 (+ 193us) server_negotiation.cc:407] Connection header received
0809 00:41:24.042470 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.042472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.042491 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.042554 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:24.043432 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.043953 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.045324 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.045522 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.045586 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.046612 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:24.046614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:24.046614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:24.046620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:24.046638 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:24.046643 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:24.046699 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.046774 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.046801 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 00:41:24.047065 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.041369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34015 (local address 127.7.105.65:33857)
0809 00:41:24.041625 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.041661 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.042355 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.042627 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.042631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.042641 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:24.043334 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.043341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.044091 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.044092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.045228 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.045231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.045613 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.045614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.045772 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.046438 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:24.046439 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:24.046468 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:24.046491 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:24.046707 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:24.046712 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:24.046715 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.046771 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.046858 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260809 00:41:24.047655 27471 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34015
I20260809 00:41:24.047768 27471 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:24.047981 27471 heartbeater.cc:511] Master 127.7.105.126:34015 requested a full tablet report, sending...
I20260809 00:41:24.048617 27266 ts_manager.cc:194] Registered new tserver with Master: ae38ea9dedf745cf805ac4ad70634a92 (127.7.105.65:43849)
I20260809 00:41:24.049633 27266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.65:33857
W20260809 00:41:24.130216 27474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:24.130419 27474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:24.130443 27474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:24.131543 27474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:24.131577 27474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:24.131605 27474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:24.132915 27474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:24.133119 27474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:24.133376 27474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:24.138533 27480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:24.138581 27482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:24.138577 27479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:24.138672 27474 server_base.cc:1034] running on GCE node
I20260809 00:41:24.139127 27474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:24.139382 27474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:24.140549 27474 hybrid_clock.cc:648] HybridClock initialized: now 1786236084140530 us; error 38 us; skew 500 ppm
I20260809 00:41:24.141831 27474 webserver.cc:460] Webserver started at http://127.7.105.66:43083/ using document root <none> and password file <none>
I20260809 00:41:24.142108 27474 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:24.142180 27474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:24.142343 27474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:24.143417 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "be2f6852f5524de1a1f32562e3c03e9c"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "0d6a44fa5202eb33680ad8e47b3f5960"
server_key_iv: "8caaa6ceedaeb572b681da8a695eeae3"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.143790 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "be2f6852f5524de1a1f32562e3c03e9c"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "0d6a44fa5202eb33680ad8e47b3f5960"
server_key_iv: "8caaa6ceedaeb572b681da8a695eeae3"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.145219 27474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:24.145983 27489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.146292 27474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:24.146435 27474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "be2f6852f5524de1a1f32562e3c03e9c"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "0d6a44fa5202eb33680ad8e47b3f5960"
server_key_iv: "8caaa6ceedaeb572b681da8a695eeae3"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.146536 27474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:24.165728 27474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:24.166115 27474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:24.166291 27474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:24.166569 27474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:24.166958 27474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:24.167028 27474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.167094 27474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:24.167128 27474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.173902 27474 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:38301
I20260809 00:41:24.173941 27608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:38301 every 8 connection(s)
I20260809 00:41:24.174347 27474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:41:24.180260 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27474
I20260809 00:41:24.180379 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:24.180405 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.175273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:45851 (local address 127.7.105.126:34015)
0809 00:41:24.175369 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.175371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.175621 (+ 250us) server_negotiation.cc:407] Connection header received
0809 00:41:24.176316 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.176317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.176333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.176401 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:24.177152 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.177672 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.178758 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.178972 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.179068 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.180043 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:24.180044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:24.180045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:24.180071 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:24.180086 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:24.180091 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:24.180190 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.180292 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.180322 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:41:24.180595 7589 external_mini_cluster.cc:1428] Setting key 27406ed07828c1194220f2ce5115734a
I20260809 00:41:24.180657 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.175231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34015 (local address 127.7.105.66:45851)
0809 00:41:24.175489 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.175525 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.176194 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.176473 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.176477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.176487 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:24.177033 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.177038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.177787 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.177788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.178622 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.178637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.179083 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.179084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.179234 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.179789 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:24.179790 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:24.179819 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:24.179843 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:24.180190 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:24.180199 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:24.180203 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.180294 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.180407 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":103,"threads_started":1}
I20260809 00:41:24.181216 27609 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34015
I20260809 00:41:24.181341 27609 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:24.181535 27609 heartbeater.cc:511] Master 127.7.105.126:34015 requested a full tablet report, sending...
I20260809 00:41:24.181706 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--builtin_ntp_servers=127.7.105.84:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:41:24.181934 27266 ts_manager.cc:194] Registered new tserver with Master: be2f6852f5524de1a1f32562e3c03e9c (127.7.105.66:38301)
I20260809 00:41:24.182330 27266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.66:45851
W20260809 00:41:24.269734 27612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:24.269944 27612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:24.269973 27612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:24.271054 27612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:24.271087 27612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:24.271111 27612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:24.272405 27612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:24.272593 27612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:24.272837 27612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:24.278053 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:24.278139 27617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:24.278192 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:24.278519 27612 server_base.cc:1034] running on GCE node
I20260809 00:41:24.278752 27612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:24.279002 27612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:24.280164 27612 hybrid_clock.cc:648] HybridClock initialized: now 1786236084280149 us; error 30 us; skew 500 ppm
I20260809 00:41:24.281754 27612 webserver.cc:460] Webserver started at http://127.7.105.67:33943/ using document root <none> and password file <none>
I20260809 00:41:24.282038 27612 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:24.282117 27612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:24.282291 27612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:24.283464 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "9b9d288fce454a0699f410f91f2a7fcc"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "701b6e491bfa19d0b5287cf1799e1e97"
server_key_iv: "1e300496a2d99d623bb02ee1705ad119"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.283865 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "9b9d288fce454a0699f410f91f2a7fcc"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "701b6e491bfa19d0b5287cf1799e1e97"
server_key_iv: "1e300496a2d99d623bb02ee1705ad119"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.285393 27612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:24.286147 27626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.286325 27612 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:41:24.286437 27612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "9b9d288fce454a0699f410f91f2a7fcc"
format_stamp: "Formatted at 2026-08-09 00:41:24 on dist-test-slave-1zqn"
server_key: "701b6e491bfa19d0b5287cf1799e1e97"
server_key_iv: "1e300496a2d99d623bb02ee1705ad119"
server_key_version: "encryptionkey@0"
I20260809 00:41:24.286516 27612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:24.302466 27612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:24.302839 27612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:24.303031 27612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:24.303295 27612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:24.303646 27612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:24.303709 27612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.303753 27612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:24.303774 27612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:24.309816 27612 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:45387
I20260809 00:41:24.309854 27748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:45387 every 8 connection(s)
I20260809 00:41:24.310249 27612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:41:24.315779 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.311065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:36821 (local address 127.7.105.126:34015)
0809 00:41:24.311156 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.311159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.311398 (+ 239us) server_negotiation.cc:407] Connection header received
0809 00:41:24.311881 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.311882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.311892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.311955 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:24.312736 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.313280 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.314284 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.314470 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.314521 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.315473 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:24.315474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:24.315474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:24.315479 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:24.315496 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:24.315501 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:24.315563 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.315693 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.315716 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:41:24.315958 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.311043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34015 (local address 127.7.105.67:36821)
0809 00:41:24.311261 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.311301 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.311795 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.312021 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.312025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.312033 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:24.312636 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.312642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.313367 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.313368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.314201 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.314204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.314549 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.314550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.314662 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.315302 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:24.315303 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:24.315333 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:24.315357 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:24.315581 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:24.315587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:24.315590 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.315657 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.315740 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260809 00:41:24.316535 27749 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34015
I20260809 00:41:24.316653 27749 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:24.316843 27749 heartbeater.cc:511] Master 127.7.105.126:34015 requested a full tablet report, sending...
I20260809 00:41:24.317255 27266 ts_manager.cc:194] Registered new tserver with Master: 9b9d288fce454a0699f410f91f2a7fcc (127.7.105.67:45387)
I20260809 00:41:24.317654 27266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.67:36821
I20260809 00:41:24.317924 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27612
I20260809 00:41:24.318008 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:24.318245 7589 external_mini_cluster.cc:1428] Setting key 5a31446331d033fa9f0256db53b434bd
I20260809 00:41:24.319216 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:24.323134 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.320611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.7.105.126:34015)
0809 00:41:24.320648 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.320650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.320773 (+ 123us) server_negotiation.cc:407] Connection header received
0809 00:41:24.320821 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.320822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.320831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.320874 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:24.321008 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.321464 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.322158 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.322334 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.322391 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.322952 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:24.322953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:24.322953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:24.322957 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:24.322969 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:24.322973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:24.323015 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.323063 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.323081 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:41:24.324848 27266 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:24.326494 27266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:24.333086 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.328770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43849 (local address 127.0.0.1:42624)
0809 00:41:24.328923 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.328959 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.329021 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.329630 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.329633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.329636 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:24.329713 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.329715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.330642 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.330643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.331800 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.331803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.332779 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.332864 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.332876 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.332947 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.333002 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":5,"thread_start_us":71,"threads_started":1}
I20260809 00:41:24.333096 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.328887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.7.105.65:43849)
0809 00:41:24.329397 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.329402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.329412 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:24.329454 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.329456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.329473 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.329547 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:24.329804 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.330546 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.331896 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.332683 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332759 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.332856 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.332963 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.333007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":105,"threads_started":1}
I20260809 00:41:24.333413 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.329232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.7.105.66:38301)
0809 00:41:24.329442 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.329447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.329458 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:41:24.329499 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.329501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.329516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.329598 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:24.329832 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.330972 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332262 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.332787 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332856 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.332962 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.333295 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.333338 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260809 00:41:24.333600 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.328962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.7.105.67:45387)
0809 00:41:24.329679 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.329684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.329694 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:24.329728 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.329730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.329744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.329955 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:24.330006 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.331046 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332203 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.332901 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.333351 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.333437 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.333459 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.333517 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":86,"threads_started":1}
I20260809 00:41:24.333238 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.328770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:45387 (local address 127.0.0.1:57344)
0809 00:41:24.329089 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.329100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.329154 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.329847 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.329847 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.329849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:24.329887 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.329888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.331105 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.331107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.332107 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.332111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.332997 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.332998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.333064 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.333068 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.333101 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.333145 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":4,"thread_start_us":51,"threads_started":1}
I20260809 00:41:24.333256 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.328959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:38301 (local address 127.0.0.1:46164)
0809 00:41:24.329219 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.329227 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.329265 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.329671 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.329672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.329674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:24.329737 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.329739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.331067 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.331068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.332168 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.332171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.333000 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.333001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.333056 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.333060 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.333107 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.333150 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260809 00:41:24.334414 27399 tablet_service.cc:1467] Processing CreateTablet for tablet cc441de2965241a6b8956c6f17b694b4 (DEFAULT_TABLE table=test-table [id=82581d7774cb4ab7b94feee4ac562f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:24.334412 27682 tablet_service.cc:1467] Processing CreateTablet for tablet cc441de2965241a6b8956c6f17b694b4 (DEFAULT_TABLE table=test-table [id=82581d7774cb4ab7b94feee4ac562f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:24.334806 27682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc441de2965241a6b8956c6f17b694b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:24.334807 27399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc441de2965241a6b8956c6f17b694b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:24.334762 27541 tablet_service.cc:1467] Processing CreateTablet for tablet cc441de2965241a6b8956c6f17b694b4 (DEFAULT_TABLE table=test-table [id=82581d7774cb4ab7b94feee4ac562f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:24.335096 27541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc441de2965241a6b8956c6f17b694b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:24.337767 27764 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Bootstrap starting.
I20260809 00:41:24.338194 27765 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Bootstrap starting.
I20260809 00:41:24.338620 27764 tablet_bootstrap.cc:654] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:24.338881 27765 tablet_bootstrap.cc:654] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:24.339025 27764 log.cc:824] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:24.339188 27765 log.cc:824] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:24.339526 27766 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Bootstrap starting.
I20260809 00:41:24.339848 27764 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: No bootstrap required, opened a new log
I20260809 00:41:24.339916 27765 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: No bootstrap required, opened a new log
I20260809 00:41:24.339933 27764 ts_tablet_manager.cc:1397] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:24.340001 27765 ts_tablet_manager.cc:1397] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:24.340236 27766 tablet_bootstrap.cc:654] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:24.340634 27766 log.cc:824] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:24.341487 27766 tablet_bootstrap.cc:492] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: No bootstrap required, opened a new log
I20260809 00:41:24.341562 27766 ts_tablet_manager.cc:1397] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:24.342262 27764 raft_consensus.cc:348] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.342453 27764 raft_consensus.cc:374] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:24.342505 27764 raft_consensus.cc:729] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be2f6852f5524de1a1f32562e3c03e9c, State: Initialized, Role: FOLLOWER
I20260809 00:41:24.342617 27764 consensus_queue.cc:260] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.342742 27765 raft_consensus.cc:348] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.342914 27764 ts_tablet_manager.cc:1428] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:41:24.342904 27765 raft_consensus.cc:374] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:24.342962 27765 raft_consensus.cc:729] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9d288fce454a0699f410f91f2a7fcc, State: Initialized, Role: FOLLOWER
I20260809 00:41:24.343010 27609 heartbeater.cc:503] Master 127.7.105.126:34015 was elected leader, sending a full tablet report...
I20260809 00:41:24.343079 27765 consensus_queue.cc:260] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.343320 27765 ts_tablet_manager.cc:1428] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 00:41:24.343676 27749 heartbeater.cc:503] Master 127.7.105.126:34015 was elected leader, sending a full tablet report...
I20260809 00:41:24.343962 27766 raft_consensus.cc:348] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.344154 27766 raft_consensus.cc:374] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:24.344209 27766 raft_consensus.cc:729] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae38ea9dedf745cf805ac4ad70634a92, State: Initialized, Role: FOLLOWER
I20260809 00:41:24.344327 27766 consensus_queue.cc:260] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.344583 27766 ts_tablet_manager.cc:1428] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:41:24.344658 27471 heartbeater.cc:503] Master 127.7.105.126:34015 was elected leader, sending a full tablet report...
W20260809 00:41:24.425005 27610 tablet.cc:2369] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:24.511647 27771 raft_consensus.cc:482] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:24.511823 27771 raft_consensus.cc:504] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.512341 27771 leader_election.cc:288] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae38ea9dedf745cf805ac4ad70634a92 (127.7.105.65:43849), 9b9d288fce454a0699f410f91f2a7fcc (127.7.105.67:45387)
I20260809 00:41:24.515865 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.512696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:34425 (local address 127.7.105.67:45387)
0809 00:41:24.512741 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.512743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.512751 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:24.512785 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.512786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.512799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.512862 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:24.513062 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.513842 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.514948 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.515494 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.515573 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.515581 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.515748 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.515795 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:24.515857 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.512533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:45387 (local address 127.7.105.66:34425)
0809 00:41:24.512606 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.512621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.512676 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.512888 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.512889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.512892 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:24.512970 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.512973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.514026 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.514027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.514846 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.514848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.515605 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.515606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.515663 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.515666 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.515713 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.515746 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":35}
I20260809 00:41:24.515967 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.512538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43849 (local address 127.7.105.66:41641)
0809 00:41:24.512769 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.512779 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:24.512846 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:24.513039 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:24.513040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:24.513042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:24.513130 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.513132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.513980 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.513980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.514847 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:24.514850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.515642 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.515643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:24.515739 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.515743 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:24.515800 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:41:24.515836 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":72,"thread_start_us":122,"threads_started":1}
I20260809 00:41:24.515969 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.512627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:41641 (local address 127.7.105.65:43849)
0809 00:41:24.512678 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.512680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.512874 (+ 194us) server_negotiation.cc:407] Connection header received
0809 00:41:24.512911 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.512911 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.512927 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.513003 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:24.513232 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.513908 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.514984 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.515505 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.515573 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.515581 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.515846 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.515900 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:41:24.516356 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc441de2965241a6b8956c6f17b694b4" candidate_uuid: "be2f6852f5524de1a1f32562e3c03e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9d288fce454a0699f410f91f2a7fcc" is_pre_election: true
I20260809 00:41:24.516409 27419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc441de2965241a6b8956c6f17b694b4" candidate_uuid: "be2f6852f5524de1a1f32562e3c03e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae38ea9dedf745cf805ac4ad70634a92" is_pre_election: true
I20260809 00:41:24.516539 27702 raft_consensus.cc:2454] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be2f6852f5524de1a1f32562e3c03e9c in term 0.
I20260809 00:41:24.516595 27419 raft_consensus.cc:2454] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be2f6852f5524de1a1f32562e3c03e9c in term 0.
I20260809 00:41:24.516805 27491 leader_election.cc:302] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9d288fce454a0699f410f91f2a7fcc, be2f6852f5524de1a1f32562e3c03e9c; no voters:
I20260809 00:41:24.516971 27771 raft_consensus.cc:2781] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:24.517050 27771 raft_consensus.cc:482] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:24.517068 27771 raft_consensus.cc:3037] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:24.517861 27771 raft_consensus.cc:504] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.518025 27771 leader_election.cc:288] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [CANDIDATE]: Term 1 election: Requested vote from peers ae38ea9dedf745cf805ac4ad70634a92 (127.7.105.65:43849), 9b9d288fce454a0699f410f91f2a7fcc (127.7.105.67:45387)
I20260809 00:41:24.518209 27419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc441de2965241a6b8956c6f17b694b4" candidate_uuid: "be2f6852f5524de1a1f32562e3c03e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae38ea9dedf745cf805ac4ad70634a92"
I20260809 00:41:24.518193 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc441de2965241a6b8956c6f17b694b4" candidate_uuid: "be2f6852f5524de1a1f32562e3c03e9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9d288fce454a0699f410f91f2a7fcc"
I20260809 00:41:24.518314 27419 raft_consensus.cc:3037] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:24.518311 27702 raft_consensus.cc:3037] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:24.519135 27419 raft_consensus.cc:2454] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be2f6852f5524de1a1f32562e3c03e9c in term 1.
I20260809 00:41:24.519366 27490 leader_election.cc:302] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae38ea9dedf745cf805ac4ad70634a92, be2f6852f5524de1a1f32562e3c03e9c; no voters:
I20260809 00:41:24.519414 27702 raft_consensus.cc:2454] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be2f6852f5524de1a1f32562e3c03e9c in term 1.
I20260809 00:41:24.519459 27771 raft_consensus.cc:2781] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:24.519618 27771 raft_consensus.cc:686] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 1 LEADER]: Becoming Leader. State: Replica: be2f6852f5524de1a1f32562e3c03e9c, State: Running, Role: LEADER
I20260809 00:41:24.519739 27771 consensus_queue.cc:237] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } }
I20260809 00:41:24.520684 27266 catalog_manager.cc:5495] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c reported cstate change: term changed from 0 to 1, leader changed from <none> to be2f6852f5524de1a1f32562e3c03e9c (127.7.105.66). New cstate: current_term: 1 leader_uuid: "be2f6852f5524de1a1f32562e3c03e9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be2f6852f5524de1a1f32562e3c03e9c" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 } health_report { overall_health: UNKNOWN } } }
W20260809 00:41:24.541339 27472 tablet.cc:2369] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:41:24.560810 27750 tablet.cc:2369] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:24.561985 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.559437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.7.105.66:38301)
0809 00:41:24.559509 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:24.559511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:24.559520 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:24.559566 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:24.559567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:24.559593 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:24.559680 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:24.559832 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.560374 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.561185 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:24.561392 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:24.561465 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:24.561627 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:24.561738 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:24.561829 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:24.561852 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 00:41:24.561872 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:41:24.563814 27419 raft_consensus.cc:1261] T cc441de2965241a6b8956c6f17b694b4 P ae38ea9dedf745cf805ac4ad70634a92 [term 1 FOLLOWER]: Refusing update from remote peer be2f6852f5524de1a1f32562e3c03e9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:24.563815 27702 raft_consensus.cc:1261] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 1 FOLLOWER]: Refusing update from remote peer be2f6852f5524de1a1f32562e3c03e9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:24.564252 27775 consensus_queue.cc:1035] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae38ea9dedf745cf805ac4ad70634a92" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 43849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:24.564406 27771 consensus_queue.cc:1035] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b9d288fce454a0699f410f91f2a7fcc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:24.566928 27779 mvcc.cc:205] Tried to move back new op lower bound from 7316423002370699264 to 7316423002193383424. Current Snapshot: MvccSnapshot[applied={T|T < 7316423002370699264}]
I20260809 00:41:24.569406 27265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:24.569517 27265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:24.570293 27265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01cfdfbfc4374105b4bee7c582e31c26: Sending DeleteTablet for 3 replicas of tablet cc441de2965241a6b8956c6f17b694b4
I20260809 00:41:24.570655 27399 tablet_service.cc:1514] Processing DeleteTablet for tablet cc441de2965241a6b8956c6f17b694b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:24 UTC) from {username='slave'} at 127.0.0.1:42624
I20260809 00:41:24.570665 27541 tablet_service.cc:1514] Processing DeleteTablet for tablet cc441de2965241a6b8956c6f17b694b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:24 UTC) from {username='slave'} at 127.0.0.1:46164
I20260809 00:41:24.570755 27682 tablet_service.cc:1514] Processing DeleteTablet for tablet cc441de2965241a6b8956c6f17b694b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:24 UTC) from {username='slave'} at 127.0.0.1:57344
I20260809 00:41:24.571060 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27330
I20260809 00:41:24.577069 27790 tablet_replica.cc:330] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: stopping tablet replica
I20260809 00:41:24.577237 27790 raft_consensus.cc:2229] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:24.577390 27790 raft_consensus.cc:2258] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:24.571015 27788 tablet_replica.cc:330] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: stopping tablet replica
I20260809 00:41:24.577627 27788 raft_consensus.cc:2229] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:41:24.577775 27788 raft_consensus.cc:2258] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:24.577979 27790 ts_tablet_manager.cc:1905] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:24.578238 27788 ts_tablet_manager.cc:1905] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:24.579852 27790 ts_tablet_manager.cc:1918] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:24.579916 27788 ts_tablet_manager.cc:1918] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:24.579943 27790 log.cc:1196] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/cc441de2965241a6b8956c6f17b694b4
I20260809 00:41:24.579986 27788 log.cc:1196] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/cc441de2965241a6b8956c6f17b694b4
I20260809 00:41:24.580265 27788 ts_tablet_manager.cc:1939] T cc441de2965241a6b8956c6f17b694b4 P be2f6852f5524de1a1f32562e3c03e9c: Deleting consensus metadata
I20260809 00:41:24.580251 27790 ts_tablet_manager.cc:1939] T cc441de2965241a6b8956c6f17b694b4 P 9b9d288fce454a0699f410f91f2a7fcc: Deleting consensus metadata
I20260809 00:41:24.580700 27248 catalog_manager.cc:4841] TS be2f6852f5524de1a1f32562e3c03e9c (127.7.105.66:38301): tablet cc441de2965241a6b8956c6f17b694b4 (table test-table [id=82581d7774cb4ab7b94feee4ac562f11]) successfully deleted
I20260809 00:41:24.580729 27250 catalog_manager.cc:4841] TS 9b9d288fce454a0699f410f91f2a7fcc (127.7.105.67:45387): tablet cc441de2965241a6b8956c6f17b694b4 (table test-table [id=82581d7774cb4ab7b94feee4ac562f11]) successfully deleted
W20260809 00:41:24.587883 27248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:43849 (error 108)
I20260809 00:41:24.588205 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27474
I20260809 00:41:24.590111 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:24.589957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:43849 (local address 127.0.0.1:42628)
0809 00:41:24.590015 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:24.590066 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:43849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:41:24.590296 27248 catalog_manager.cc:4584] TS ae38ea9dedf745cf805ac4ad70634a92 (127.7.105.65:43849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc441de2965241a6b8956c6f17b694b4: Network error: Client connection negotiation failed: client connection to 127.7.105.65:43849: connect: Connection refused (error 111)
I20260809 00:41:24.595490 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27612
I20260809 00:41:24.602733 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27233
2026-08-09T00:41:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:24 dist-test-slave-1zqn krb5kdc[27795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:24 dist-test-slave-1zqn krb5kdc[27795](info): set up 2 sockets
Aug 09 00:41:24 dist-test-slave-1zqn krb5kdc[27795](info): commencing operation
krb5kdc: starting...
W20260809 00:41:25.731139 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:25 dist-test-slave-1zqn krb5kdc[27795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:25.861014 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34809
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:34809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:25.945710 27809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:25.945959 27809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:25.945996 27809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:25.947121 27809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:25.947192 27809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:25.947216 27809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:25.947235 27809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:25.947268 27809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:25.948769 27809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:34809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:25.949051 27809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:25.949327 27809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:25.954421 27819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:25.954439 27816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:25.954514 27814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:25.954749 27809 server_base.cc:1034] running on GCE node
I20260809 00:41:25.954986 27809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:25.955276 27809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:25.956447 27809 hybrid_clock.cc:648] HybridClock initialized: now 1786236085956421 us; error 53 us; skew 500 ppm
Aug 09 00:41:25 dist-test-slave-1zqn krb5kdc[27795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236085, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:25.959069 27809 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:25.959522 27809 webserver.cc:460] Webserver started at http://127.7.105.126:44571/ using document root <none> and password file <none>
I20260809 00:41:25.959755 27809 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:25.959825 27809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:25.959986 27809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:25.961072 27809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "75a1b2cb3c3244e4b7daeca501419959"
format_stamp: "Formatted at 2026-08-09 00:41:25 on dist-test-slave-1zqn"
server_key: "ea2239b5b546a43ce596dac1f1cfdbb0"
server_key_iv: "b7cfb33783971b03d8c76f8832a1188c"
server_key_version: "encryptionkey@0"
I20260809 00:41:25.961442 27809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "75a1b2cb3c3244e4b7daeca501419959"
format_stamp: "Formatted at 2026-08-09 00:41:25 on dist-test-slave-1zqn"
server_key: "ea2239b5b546a43ce596dac1f1cfdbb0"
server_key_iv: "b7cfb33783971b03d8c76f8832a1188c"
server_key_version: "encryptionkey@0"
I20260809 00:41:25.962926 27809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:41:25.963719 27827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:25.963907 27809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:25.964010 27809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "75a1b2cb3c3244e4b7daeca501419959"
format_stamp: "Formatted at 2026-08-09 00:41:25 on dist-test-slave-1zqn"
server_key: "ea2239b5b546a43ce596dac1f1cfdbb0"
server_key_iv: "b7cfb33783971b03d8c76f8832a1188c"
server_key_version: "encryptionkey@0"
I20260809 00:41:25.964146 27809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:25.988257 27809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:25.990665 27809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:25.990870 27809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:25.995174 27809 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:34809
I20260809 00:41:25.995215 27882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:34809 every 8 connection(s)
I20260809 00:41:25.995589 27809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:25.996737 27883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:25.997041 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27809
I20260809 00:41:25.997202 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:25.997407 7589 external_mini_cluster.cc:1428] Setting key c008139f9f6c8e16cfbcf0ebdbe5f19a
I20260809 00:41:26.000042 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: Bootstrap starting.
I20260809 00:41:26.000802 27883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:26.001125 27883 log.cc:824] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:26.001905 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: No bootstrap required, opened a new log
I20260809 00:41:26.004590 27883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } }
I20260809 00:41:26.004735 27883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:26.004758 27883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a1b2cb3c3244e4b7daeca501419959, State: Initialized, Role: FOLLOWER
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:26.004899 27883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } }
I20260809 00:41:26.004995 27883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:26.005050 27883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:26.005102 27883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:26.005893 27883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } }
I20260809 00:41:26.006018 27883 leader_election.cc:302] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a1b2cb3c3244e4b7daeca501419959; no voters:
I20260809 00:41:26.006191 27883 leader_election.cc:288] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:26.006322 27888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:26.006500 27883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:26.006886 27888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [term 1 LEADER]: Becoming Leader. State: Replica: 75a1b2cb3c3244e4b7daeca501419959, State: Running, Role: LEADER
I20260809 00:41:26.008170 27888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } }
I20260809 00:41:26.008859 27889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a1b2cb3c3244e4b7daeca501419959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } } }
I20260809 00:41:26.008967 27889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:26.009338 27890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a1b2cb3c3244e4b7daeca501419959. Latest consensus state: current_term: 1 leader_uuid: "75a1b2cb3c3244e4b7daeca501419959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a1b2cb3c3244e4b7daeca501419959" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34809 } } }
I20260809 00:41:26.009449 27890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959 [sys.catalog]: This master's current role is: LEADER
W20260809 00:41:26.009336 27903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:26.009545 27903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:26.009593 27904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:26.009827 27904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:26.010479 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:25.998014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.7.105.126:34809)
0809 00:41:25.998214 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:25.998222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:25.998248 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:41:25.998701 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:25.998708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:25.998757 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:25.998874 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:25.999708 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.000719 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.001746 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.002302 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.002392 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.005561 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.005566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.005566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:26.005579 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.008076 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.008511 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.008515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.008515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.008655 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.008937 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.008939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.008939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.009343 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.009492 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.010101 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.010265 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260809 00:41:26.011729 27904 catalog_manager.cc:1347] Generated new cluster ID: 960a4dcf2e3c453c84c9a80fbf33ce1b
I20260809 00:41:26.011785 27904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:26.016882 27904 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:26.017562 27904 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:26.022814 27904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: Generated new TSK 0
I20260809 00:41:26.023008 27904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:41:26.055465 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:26.143975 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:26.144264 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:26.144309 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:26.145510 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:26.145584 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:26.145648 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:26.147228 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:26.147523 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:26.147784 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:26.153530 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.153530 27916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.153613 27917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:26.153635 27911 server_base.cc:1034] running on GCE node
I20260809 00:41:26.153997 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:26.154292 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:26.155465 27911 hybrid_clock.cc:648] HybridClock initialized: now 1786236086155457 us; error 49 us; skew 500 ppm
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:26.158311 27911 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:41:26.158846 27911 webserver.cc:460] Webserver started at http://127.7.105.65:37247/ using document root <none> and password file <none>
I20260809 00:41:26.159111 27911 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:26.159183 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:26.159337 27911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:26.160493 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "a9c9c3b65d584d73a9e848c586793f49"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fc8845fdba19389a0ebe37e7f171470e"
server_key_iv: "062cdd11d5e8d42bd7499dfc204fb017"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.160867 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "a9c9c3b65d584d73a9e848c586793f49"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fc8845fdba19389a0ebe37e7f171470e"
server_key_iv: "062cdd11d5e8d42bd7499dfc204fb017"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.162274 27911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:41:26.163136 27927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.163360 27911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:26.163475 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "a9c9c3b65d584d73a9e848c586793f49"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fc8845fdba19389a0ebe37e7f171470e"
server_key_iv: "062cdd11d5e8d42bd7499dfc204fb017"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.163551 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:26.194877 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:26.197506 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:26.197695 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:26.197964 27911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:26.198311 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:26.198366 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.198410 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:26.198457 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.204856 27911 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:33031
I20260809 00:41:26.204903 28050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:33031 every 8 connection(s)
I20260809 00:41:26.205238 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:26.212118 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 27911
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:26.212219 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:26.212426 7589 external_mini_cluster.cc:1428] Setting key d6a26fd7903312b024941dcddb5b6d24
I20260809 00:41:26.216863 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.206089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34809 (local address 127.7.105.65:53421)
0809 00:41:26.206234 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.206273 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.206861 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.207131 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.207135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.207802 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:26.208027 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.208032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.208859 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.208860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.209672 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.209674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.210115 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.210115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.210222 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.210764 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:26.210772 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:26.212712 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:26.215447 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.215454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.215456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.215689 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.215906 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.215907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.215907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.215993 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.216320 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:26.216326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:26.216448 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.216559 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.216687 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260809 00:41:26.216866 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.206035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:53421 (local address 127.7.105.126:34809)
0809 00:41:26.206111 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.206113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.206350 (+ 237us) server_negotiation.cc:407] Connection header received
0809 00:41:26.206994 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.206996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.207009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.207074 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:26.208169 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.208759 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.209776 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.210026 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.210094 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.212804 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.212821 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.212821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:26.212831 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.215366 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.215769 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.215770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.215770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.215842 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.216084 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.216085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.216085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.216249 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.216324 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.216628 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.216727 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:41:26.217388 28051 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34809
I20260809 00:41:26.217500 28051 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:26.217715 28051 heartbeater.cc:511] Master 127.7.105.126:34809 requested a full tablet report, sending...
I20260809 00:41:26.218403 27845 ts_manager.cc:194] Registered new tserver with Master: a9c9c3b65d584d73a9e848c586793f49 (127.7.105.65:33031)
I20260809 00:41:26.219300 27845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:53421
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:41:26.240899 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:26.327348 28056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:26.327585 28056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:26.327625 28056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:26.328801 28056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:26.328871 28056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:26.328917 28056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:26.330369 28056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:26.330644 28056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:26.330894 28056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:26.336355 28064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.336395 28062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.336328 28066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:26.336530 28056 server_base.cc:1034] running on GCE node
I20260809 00:41:26.336742 28056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:26.336951 28056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:26.338092 28056 hybrid_clock.cc:648] HybridClock initialized: now 1786236086338080 us; error 33 us; skew 500 ppm
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:26.340894 28056 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:41:26.341339 28056 webserver.cc:460] Webserver started at http://127.7.105.66:37051/ using document root <none> and password file <none>
I20260809 00:41:26.341565 28056 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:26.341619 28056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:26.341747 28056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:26.342836 28056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "bcde2fd7d41dcdbd578eaee88fc2bc85"
server_key_iv: "fe5beba737b655aaa2aba1271c2e7e79"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.343226 28056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "bcde2fd7d41dcdbd578eaee88fc2bc85"
server_key_iv: "fe5beba737b655aaa2aba1271c2e7e79"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.344702 28056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:26.345481 28074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.345660 28056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:26.345770 28056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "bcde2fd7d41dcdbd578eaee88fc2bc85"
server_key_iv: "fe5beba737b655aaa2aba1271c2e7e79"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.345847 28056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:26.358265 28056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:26.360764 28056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:26.360947 28056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:26.361220 28056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:26.361547 28056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:26.361603 28056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.361649 28056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:26.361678 28056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.368098 28056 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39733
I20260809 00:41:26.368158 28198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39733 every 8 connection(s)
I20260809 00:41:26.368510 28056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:26.377969 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.369304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:49335 (local address 127.7.105.126:34809)
0809 00:41:26.369397 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.369399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.369578 (+ 179us) server_negotiation.cc:407] Connection header received
0809 00:41:26.370093 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.370094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.370104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.370159 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:26.371152 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.371651 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.372689 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.372872 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.372928 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.375162 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.375164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.375164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:26.375170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.376622 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.377095 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.377096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.377097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.377161 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.377430 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.377430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.377430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.377533 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.377583 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.377837 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.377901 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:41:26.378296 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.369262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34809 (local address 127.7.105.66:49335)
0809 00:41:26.369430 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.369469 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.369963 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.370246 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.370250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.370854 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:26.371054 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.371058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.371731 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.371732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.372609 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.372612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.372949 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.372950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.373049 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.373540 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:26.373548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:26.374958 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:26.376703 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.376711 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.376713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.377022 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.377242 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.377243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.377243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.377359 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.377593 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:26.377598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:26.377722 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.377810 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.377936 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260809 00:41:26.378412 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28056
I20260809 00:41:26.378526 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:26.378800 7589 external_mini_cluster.cc:1428] Setting key 96f405fdfe37e7977da484c2a5e896af
I20260809 00:41:26.378933 28199 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34809
I20260809 00:41:26.379065 28199 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:26.379269 28199 heartbeater.cc:511] Master 127.7.105.126:34809 requested a full tablet report, sending...
I20260809 00:41:26.379683 27845 ts_manager.cc:194] Registered new tserver with Master: 130ecb1f5f874a90b3a4a4aec62bdc6a (127.7.105.66:39733)
I20260809 00:41:26.380204 27845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:49335
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:41:26.408456 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:26.494481 28204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:26.494745 28204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:26.494791 28204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:26.495977 28204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:26.496057 28204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:26.496143 28204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:26.497704 28204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:34809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:26.497996 28204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:26.498267 28204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:26.503911 28210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.503890 28212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:26.504220 28204 server_base.cc:1034] running on GCE node
W20260809 00:41:26.503886 28209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:26.504535 28204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:26.504751 28204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:26.505920 28204 hybrid_clock.cc:648] HybridClock initialized: now 1786236086505904 us; error 28 us; skew 500 ppm
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:26.508876 28204 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:41:26.509442 28204 webserver.cc:460] Webserver started at http://127.7.105.67:43133/ using document root <none> and password file <none>
I20260809 00:41:26.509783 28204 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:26.509865 28204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:26.510043 28204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:26.511165 28204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "a21491be56e54a99b640c2ad6157c4dc"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fd15d35acc7af388fd170e36eb1d957b"
server_key_iv: "a029b49033a9b356aa79b9b404723e20"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.511574 28204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "a21491be56e54a99b640c2ad6157c4dc"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fd15d35acc7af388fd170e36eb1d957b"
server_key_iv: "a029b49033a9b356aa79b9b404723e20"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.513128 28204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:26.513953 28220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.514176 28204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:26.514287 28204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "a21491be56e54a99b640c2ad6157c4dc"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "fd15d35acc7af388fd170e36eb1d957b"
server_key_iv: "a029b49033a9b356aa79b9b404723e20"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.514391 28204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:26.535221 28204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:26.537691 28204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:26.537876 28204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:26.538161 28204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:26.538515 28204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:26.538574 28204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.538635 28204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:26.538669 28204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.545723 28204 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:34679
I20260809 00:41:26.545779 28345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:34679 every 8 connection(s)
I20260809 00:41:26.546145 28204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:41:26 dist-test-slave-1zqn krb5kdc[27795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236086, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:26.554659 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28204
I20260809 00:41:26.554773 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:26.554976 7589 external_mini_cluster.cc:1428] Setting key d73ff970e650d9a2d73d241cc137bf51
I20260809 00:41:26.556602 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.546997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:52239 (local address 127.7.105.126:34809)
0809 00:41:26.547078 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.547081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.547255 (+ 174us) server_negotiation.cc:407] Connection header received
0809 00:41:26.547778 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.547779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.547789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.547857 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:26.549030 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.549618 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.550725 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.550945 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.551059 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.553493 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.553495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.553495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:26.553502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.554975 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.555533 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.555534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.555535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.555604 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.555942 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.555943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.555944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.556120 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.556187 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.556460 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.556535 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:41:26.556769 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.546907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:34809 (local address 127.7.105.67:52239)
0809 00:41:26.547102 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.547146 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.547683 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.547927 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.547931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.548600 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:26.548866 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.548915 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.549728 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.549729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.550632 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.550635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.551051 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.551052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.551219 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.551795 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:26.551803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:26.553388 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:26.555078 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.555086 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.555088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.555456 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.555683 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:26.555684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:26.555684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:26.555850 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:26.556190 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:26.556197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:26.556318 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.556401 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.556526 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260809 00:41:26.557338 28346 heartbeater.cc:347] Connected to a master server at 127.7.105.126:34809
I20260809 00:41:26.557423 28346 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:26.557565 28346 heartbeater.cc:511] Master 127.7.105.126:34809 requested a full tablet report, sending...
I20260809 00:41:26.557890 27845 ts_manager.cc:194] Registered new tserver with Master: a21491be56e54a99b640c2ad6157c4dc (127.7.105.67:34679)
I20260809 00:41:26.558276 27845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:52239
I20260809 00:41:26.566926 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:26.574820 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.568507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.7.105.126:34809)
0809 00:41:26.568597 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.568600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.568786 (+ 186us) server_negotiation.cc:407] Connection header received
0809 00:41:26.568833 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.568834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.568846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.568904 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:26.569695 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.570168 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.570849 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.571104 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.571163 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.572574 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.572575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.572576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:26.572582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.573773 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.574136 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.574137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.574137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.574190 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:26.574377 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:26.574378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:26.574378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:26.574510 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.574593 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.574708 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.574756 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:26.576411 27845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:26.577988 27845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:26.584935 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39733 (local address 127.0.0.1:57382)
0809 00:41:26.580773 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.580813 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.580878 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.581288 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.581292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.581295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:26.581384 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.581386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582748 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.582749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.583825 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.583828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.584586 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.584587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.584679 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.584693 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.584787 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.584844 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":51,"thread_start_us":62,"threads_started":1}
I20260809 00:41:26.585049 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.7.105.67:34679)
0809 00:41:26.581423 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.581428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.581578 (+ 150us) server_negotiation.cc:407] Connection header received
0809 00:41:26.581623 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.581625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.581640 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.581724 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:26.581805 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582818 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.583908 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.584691 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.584773 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.584834 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.584913 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.584954 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":79,"threads_started":1}
I20260809 00:41:26.585179 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:34679 (local address 127.0.0.1:47808)
0809 00:41:26.581435 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.581446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.581500 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.581721 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.581721 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.581723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:26.581755 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.581756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582901 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.582902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.583820 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.583822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.584779 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.584780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.584836 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.584839 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.584886 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.584920 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":79,"threads_started":1}
I20260809 00:41:26.585482 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.7.105.66:39733)
0809 00:41:26.581116 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.581120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.581129 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:26.581164 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.581165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.581178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.581282 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:26.581476 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582632 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.583910 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.584494 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.584584 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.584657 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.585334 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.585382 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260809 00:41:26.586093 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:33031 (local address 127.0.0.1:58850)
0809 00:41:26.580933 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.580943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.580986 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.581526 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.581527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.581530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:26.581583 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.581584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582514 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.582515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.584754 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.584757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.585835 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.585836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.585904 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.585907 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.585969 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.586013 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":1077,"thread_start_us":81,"threads_started":1}
I20260809 00:41:26.586097 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.580706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.7.105.65:33031)
0809 00:41:26.581134 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.581138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.581147 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:26.581178 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.581180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.581230 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.581307 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:26.581643 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.582389 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.584848 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.585760 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.585832 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.585888 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.585980 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.586019 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":99,"threads_started":1}
I20260809 00:41:26.586084 28275 tablet_service.cc:1467] Processing CreateTablet for tablet ae0561861f724b148bc3d505e37b7875 (DEFAULT_TABLE table=test-table [id=351a967caa754e53ae958022d03a59b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:26.586305 28128 tablet_service.cc:1467] Processing CreateTablet for tablet ae0561861f724b148bc3d505e37b7875 (DEFAULT_TABLE table=test-table [id=351a967caa754e53ae958022d03a59b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:26.586517 28275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0561861f724b148bc3d505e37b7875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:26.586616 28128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0561861f724b148bc3d505e37b7875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:26.586781 27983 tablet_service.cc:1467] Processing CreateTablet for tablet ae0561861f724b148bc3d505e37b7875 (DEFAULT_TABLE table=test-table [id=351a967caa754e53ae958022d03a59b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:26.587085 27983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0561861f724b148bc3d505e37b7875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:26.589486 28365 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Bootstrap starting.
I20260809 00:41:26.590075 28366 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Bootstrap starting.
I20260809 00:41:26.590279 28365 tablet_bootstrap.cc:654] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:26.590500 28367 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Bootstrap starting.
I20260809 00:41:26.590772 28366 tablet_bootstrap.cc:654] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:26.591173 28367 tablet_bootstrap.cc:654] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:26.591352 28365 log.cc:824] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:26.591437 28366 log.cc:824] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:26.591802 28367 log.cc:824] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:26.592250 28366 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: No bootstrap required, opened a new log
I20260809 00:41:26.592327 28366 ts_tablet_manager.cc:1397] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:26.592588 28367 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: No bootstrap required, opened a new log
I20260809 00:41:26.592679 28367 ts_tablet_manager.cc:1397] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:26.593758 28366 raft_consensus.cc:348] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.593902 28366 raft_consensus.cc:374] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:26.593924 28366 raft_consensus.cc:729] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21491be56e54a99b640c2ad6157c4dc, State: Initialized, Role: FOLLOWER
I20260809 00:41:26.594053 28366 consensus_queue.cc:260] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.594141 28367 raft_consensus.cc:348] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.594349 28367 raft_consensus.cc:374] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:26.594378 28366 ts_tablet_manager.cc:1428] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:26.594398 28367 raft_consensus.cc:729] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c9c3b65d584d73a9e848c586793f49, State: Initialized, Role: FOLLOWER
I20260809 00:41:26.594391 28346 heartbeater.cc:503] Master 127.7.105.126:34809 was elected leader, sending a full tablet report...
I20260809 00:41:26.594514 28367 consensus_queue.cc:260] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.594811 28367 ts_tablet_manager.cc:1428] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:26.594938 28051 heartbeater.cc:503] Master 127.7.105.126:34809 was elected leader, sending a full tablet report...
I20260809 00:41:26.595039 28365 tablet_bootstrap.cc:492] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: No bootstrap required, opened a new log
I20260809 00:41:26.595109 28365 ts_tablet_manager.cc:1397] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 00:41:26.596885 28365 raft_consensus.cc:348] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.597040 28365 raft_consensus.cc:374] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:26.597087 28365 raft_consensus.cc:729] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130ecb1f5f874a90b3a4a4aec62bdc6a, State: Initialized, Role: FOLLOWER
I20260809 00:41:26.597204 28365 consensus_queue.cc:260] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.597474 28365 ts_tablet_manager.cc:1428] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:41:26.597549 28199 heartbeater.cc:503] Master 127.7.105.126:34809 was elected leader, sending a full tablet report...
W20260809 00:41:26.619210 28200 tablet.cc:2369] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:26.651798 28376 raft_consensus.cc:482] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:26.651969 28376 raft_consensus.cc:504] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.652487 28376 leader_election.cc:288] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9c9c3b65d584d73a9e848c586793f49 (127.7.105.65:33031), a21491be56e54a99b640c2ad6157c4dc (127.7.105.67:34679)
I20260809 00:41:26.656704 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.652752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:60875 (local address 127.7.105.65:33031)
0809 00:41:26.652803 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.652805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.652903 (+ 98us) server_negotiation.cc:407] Connection header received
0809 00:41:26.652968 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.652969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.652978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.653020 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:26.653236 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.654028 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.655658 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.656288 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.656372 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.656384 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.656579 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.656634 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:26.656711 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.652687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:33031 (local address 127.7.105.66:60875)
0809 00:41:26.652818 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.652833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.652897 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.653056 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.653057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.653059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:26.653152 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.653154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.654162 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.654164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.655525 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.655528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.656392 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.656393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.656484 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.656488 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.656560 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.656618 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":100}
I20260809 00:41:26.656759 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.652889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:39867 (local address 127.7.105.67:34679)
0809 00:41:26.652939 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.652941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.653093 (+ 152us) server_negotiation.cc:407] Connection header received
0809 00:41:26.653134 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.653135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.653142 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.653203 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:26.653322 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.654013 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.655658 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.656353 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.656413 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.656420 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.656642 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.656685 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:26.657001 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.652739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:34679 (local address 127.7.105.66:39867)
0809 00:41:26.652997 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.653025 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:26.653086 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:26.653191 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:26.653192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:26.653193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:26.653250 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.653251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.654162 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.654164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.655525 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:26.655528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.656455 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.656456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:26.656527 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.656551 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:41:26.656868 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 00:41:26.656914 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":42,"thread_start_us":126,"threads_started":1}
I20260809 00:41:26.657122 28005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0561861f724b148bc3d505e37b7875" candidate_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c9c3b65d584d73a9e848c586793f49" is_pre_election: true
I20260809 00:41:26.657308 28005 raft_consensus.cc:2454] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130ecb1f5f874a90b3a4a4aec62bdc6a in term 0.
I20260809 00:41:26.657338 28296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0561861f724b148bc3d505e37b7875" candidate_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21491be56e54a99b640c2ad6157c4dc" is_pre_election: true
I20260809 00:41:26.657495 28296 raft_consensus.cc:2454] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130ecb1f5f874a90b3a4a4aec62bdc6a in term 0.
I20260809 00:41:26.657570 28079 leader_election.cc:302] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130ecb1f5f874a90b3a4a4aec62bdc6a, a9c9c3b65d584d73a9e848c586793f49; no voters:
I20260809 00:41:26.657725 28376 raft_consensus.cc:2781] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:26.657779 28376 raft_consensus.cc:482] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:26.657805 28376 raft_consensus.cc:3037] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:26.658658 28376 raft_consensus.cc:504] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.658810 28376 leader_election.cc:288] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [CANDIDATE]: Term 1 election: Requested vote from peers a9c9c3b65d584d73a9e848c586793f49 (127.7.105.65:33031), a21491be56e54a99b640c2ad6157c4dc (127.7.105.67:34679)
I20260809 00:41:26.658965 28005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0561861f724b148bc3d505e37b7875" candidate_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c9c3b65d584d73a9e848c586793f49"
I20260809 00:41:26.658975 28296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0561861f724b148bc3d505e37b7875" candidate_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21491be56e54a99b640c2ad6157c4dc"
I20260809 00:41:26.659058 28005 raft_consensus.cc:3037] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:26.659080 28296 raft_consensus.cc:3037] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:26.659802 28296 raft_consensus.cc:2454] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130ecb1f5f874a90b3a4a4aec62bdc6a in term 1.
I20260809 00:41:26.659817 28005 raft_consensus.cc:2454] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130ecb1f5f874a90b3a4a4aec62bdc6a in term 1.
I20260809 00:41:26.660000 28079 leader_election.cc:302] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130ecb1f5f874a90b3a4a4aec62bdc6a, a9c9c3b65d584d73a9e848c586793f49; no voters:
I20260809 00:41:26.660125 28376 raft_consensus.cc:2781] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:26.660303 28376 raft_consensus.cc:686] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 1 LEADER]: Becoming Leader. State: Replica: 130ecb1f5f874a90b3a4a4aec62bdc6a, State: Running, Role: LEADER
I20260809 00:41:26.660426 28376 consensus_queue.cc:237] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } }
I20260809 00:41:26.661269 27845 catalog_manager.cc:5495] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 130ecb1f5f874a90b3a4a4aec62bdc6a (127.7.105.66). New cstate: current_term: 1 leader_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130ecb1f5f874a90b3a4a4aec62bdc6a" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 } health_report { overall_health: UNKNOWN } } }
I20260809 00:41:26.679865 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.677574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.7.105.66:39733)
0809 00:41:26.677635 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.677637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.677646 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:26.677700 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.677701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.677721 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.677782 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:26.677950 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.678423 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.679189 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.679366 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.679415 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.679568 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:26.679654 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:26.679724 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.679776 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.679786 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:41:26.681738 28296 raft_consensus.cc:1261] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 1 FOLLOWER]: Refusing update from remote peer 130ecb1f5f874a90b3a4a4aec62bdc6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:26.681749 28005 raft_consensus.cc:1261] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 1 FOLLOWER]: Refusing update from remote peer 130ecb1f5f874a90b3a4a4aec62bdc6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:26.682147 28378 consensus_queue.cc:1035] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c9c3b65d584d73a9e848c586793f49" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:26.682284 28376 consensus_queue.cc:1035] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a21491be56e54a99b640c2ad6157c4dc" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 34679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:26.684679 28382 mvcc.cc:205] Tried to move back new op lower bound from 7316423011045761024 to 7316423010961719296. Current Snapshot: MvccSnapshot[applied={T|T < 7316423011045761024}]
I20260809 00:41:26.686043 28383 mvcc.cc:205] Tried to move back new op lower bound from 7316423011045761024 to 7316423010961719296. Current Snapshot: MvccSnapshot[applied={T|T < 7316423011045761024 or (T in {7316423011045761024})}]
I20260809 00:41:26.686170 28384 mvcc.cc:205] Tried to move back new op lower bound from 7316423011045761024 to 7316423010961719296. Current Snapshot: MvccSnapshot[applied={T|T < 7316423011045761024 or (T in {7316423011045761024})}]
I20260809 00:41:26.687363 27844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:26.687464 27844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:26.688261 27844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75a1b2cb3c3244e4b7daeca501419959: Sending DeleteTablet for 3 replicas of tablet ae0561861f724b148bc3d505e37b7875
I20260809 00:41:26.688647 28128 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0561861f724b148bc3d505e37b7875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:26 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260809 00:41:26.688683 27983 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0561861f724b148bc3d505e37b7875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:26 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:58850
I20260809 00:41:26.688916 28275 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0561861f724b148bc3d505e37b7875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:26 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:47808
I20260809 00:41:26.688939 28392 tablet_replica.cc:330] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: stopping tablet replica
I20260809 00:41:26.689038 28392 raft_consensus.cc:2229] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:26.689143 28392 raft_consensus.cc:2258] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:26.689234 28393 tablet_replica.cc:330] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: stopping tablet replica
I20260809 00:41:26.689375 28393 raft_consensus.cc:2229] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:26.689486 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27911
I20260809 00:41:26.689596 28392 ts_tablet_manager.cc:1905] T ae0561861f724b148bc3d505e37b7875 P a9c9c3b65d584d73a9e848c586793f49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:26.689579 28393 raft_consensus.cc:2258] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:26.696410 28391 tablet_replica.cc:330] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: stopping tablet replica
I20260809 00:41:26.696528 28391 raft_consensus.cc:2229] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:41:26.696525 28393 ts_tablet_manager.cc:1905] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:26.696631 28391 raft_consensus.cc:2258] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:26.697068 28391 ts_tablet_manager.cc:1905] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:41:26.698128 27831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:33031 (error 108)
I20260809 00:41:26.698601 28393 ts_tablet_manager.cc:1918] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:26.698657 28391 ts_tablet_manager.cc:1918] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:26.698711 28393 log.cc:1196] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/ae0561861f724b148bc3d505e37b7875
I20260809 00:41:26.698724 28391 log.cc:1196] T ae0561861f724b148bc3d505e37b7875 P 130ecb1f5f874a90b3a4a4aec62bdc6a: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/ae0561861f724b148bc3d505e37b7875
I20260809 00:41:26.698936 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28056
I20260809 00:41:26.700341 28393 ts_tablet_manager.cc:1939] T ae0561861f724b148bc3d505e37b7875 P a21491be56e54a99b640c2ad6157c4dc: Deleting consensus metadata
I20260809 00:41:26.700801 27830 catalog_manager.cc:4841] TS a21491be56e54a99b640c2ad6157c4dc (127.7.105.67:34679): tablet ae0561861f724b148bc3d505e37b7875 (table test-table [id=351a967caa754e53ae958022d03a59b8]) successfully deleted
I20260809 00:41:26.701398 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.701238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:33031 (local address 127.0.0.1:58854)
0809 00:41:26.701302 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.701356 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:33031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 00:41:26.701735 27831 catalog_manager.cc:4584] TS a9c9c3b65d584d73a9e848c586793f49 (127.7.105.65:33031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae0561861f724b148bc3d505e37b7875: Network error: Client connection negotiation failed: client connection to 127.7.105.65:33031: connect: Connection refused (error 111)
I20260809 00:41:26.705502 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28204
I20260809 00:41:26.705535 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.705409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39733 (local address 127.0.0.1:57406)
0809 00:41:26.705478 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:26.705507 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.66:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:41:26.715683 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 27809
2026-08-09T00:41:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2115 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T00:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:26Z Disabled control of system clock
I20260809 00:41:26.749488 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43401
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:26.832222 28398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:26.832456 28398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:26.832477 28398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:26.833714 28398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:26.833760 28398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:26.833773 28398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:26.833786 28398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:26.833827 28398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:26.835173 28398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:26.835438 28398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:26.835748 28398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:26.840929 28406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.840978 28404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:26.840932 28403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:26.841074 28398 server_base.cc:1034] running on GCE node
I20260809 00:41:26.841480 28398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:26.841733 28398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:26.842886 28398 hybrid_clock.cc:648] HybridClock initialized: now 1786236086842863 us; error 38 us; skew 500 ppm
I20260809 00:41:26.848644 28398 webserver.cc:460] Webserver started at http://127.7.105.126:43103/ using document root <none> and password file <none>
I20260809 00:41:26.848917 28398 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:26.849036 28398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:26.849229 28398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:26.850342 28398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f00c493f636847fe80aadf918b4ac2ab"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "757ca69382728449c12c30af180ff442"
server_key_iv: "b80ad92c31644c8104dcc485ea8a0720"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.850729 28398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f00c493f636847fe80aadf918b4ac2ab"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "757ca69382728449c12c30af180ff442"
server_key_iv: "b80ad92c31644c8104dcc485ea8a0720"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.852174 28398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:26.852984 28414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:26.853163 28398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:26.853276 28398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f00c493f636847fe80aadf918b4ac2ab"
format_stamp: "Formatted at 2026-08-09 00:41:26 on dist-test-slave-1zqn"
server_key: "757ca69382728449c12c30af180ff442"
server_key_iv: "b80ad92c31644c8104dcc485ea8a0720"
server_key_version: "encryptionkey@0"
I20260809 00:41:26.853353 28398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:26.866963 28398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:26.867316 28398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:26.867503 28398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:26.871702 28398 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:43401
I20260809 00:41:26.871767 28469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:43401 every 8 connection(s)
I20260809 00:41:26.872159 28398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:26.873143 28470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:26.874239 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28398
I20260809 00:41:26.874359 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:26.874567 7589 external_mini_cluster.cc:1428] Setting key 5f568cb9a858ae63eb061a853225de68
I20260809 00:41:26.875886 28470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: Bootstrap starting.
I20260809 00:41:26.876616 28470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:26.876936 28470 log.cc:824] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:26.877750 28470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: No bootstrap required, opened a new log
I20260809 00:41:26.880263 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:26.875126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.7.105.126:43401)
0809 00:41:26.875309 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:26.875316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:26.875343 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:41:26.875750 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:26.875756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:26.875827 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:26.875954 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:26.876217 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.877223 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.878258 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:26.878676 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:26.878769 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:26.879521 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:26.879525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:26.879525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:26.879705 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:26.879738 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:26.879748 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:26.879800 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:26.879944 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 00:41:26.880035 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260809 00:41:26.880201 28470 raft_consensus.cc:348] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } }
I20260809 00:41:26.880329 28470 raft_consensus.cc:374] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:26.880375 28470 raft_consensus.cc:729] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00c493f636847fe80aadf918b4ac2ab, State: Initialized, Role: FOLLOWER
I20260809 00:41:26.880501 28470 consensus_queue.cc:260] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } }
I20260809 00:41:26.880579 28470 raft_consensus.cc:388] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:26.880626 28470 raft_consensus.cc:482] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:26.880677 28470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:26.881301 28470 raft_consensus.cc:504] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } }
I20260809 00:41:26.881393 28470 leader_election.cc:302] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f00c493f636847fe80aadf918b4ac2ab; no voters:
I20260809 00:41:26.881558 28470 leader_election.cc:288] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:26.881587 28475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:26.881713 28475 raft_consensus.cc:686] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [term 1 LEADER]: Becoming Leader. State: Replica: f00c493f636847fe80aadf918b4ac2ab, State: Running, Role: LEADER
I20260809 00:41:26.881799 28475 consensus_queue.cc:237] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } }
I20260809 00:41:26.881852 28470 sys_catalog.cc:564] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:26.882121 28475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f00c493f636847fe80aadf918b4ac2ab. Latest consensus state: current_term: 1 leader_uuid: "f00c493f636847fe80aadf918b4ac2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } } }
I20260809 00:41:26.882169 28477 sys_catalog.cc:455] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f00c493f636847fe80aadf918b4ac2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00c493f636847fe80aadf918b4ac2ab" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43401 } } }
I20260809 00:41:26.882197 28475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:26.882309 28477 sys_catalog.cc:458] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:26.882506 28481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:26.883133 28481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:26.884816 28481 catalog_manager.cc:1347] Generated new cluster ID: f21b5685cf904848a8e6070d2c728df9
I20260809 00:41:26.884872 28481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:26.890874 28481 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:26.891427 28481 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:26.902623 28481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: Generated new TSK 0
I20260809 00:41:26.902848 28481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:26.906845 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--builtin_ntp_servers=127.7.105.84:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:27.021349 28494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:27.021601 28494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:27.021629 28494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:27.022935 28494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:27.022989 28494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:27.023031 28494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:27.024665 28494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:27.024977 28494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:27.025372 28494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:27.031649 28502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.031646 28499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.031776 28500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:27.031785 28494 server_base.cc:1034] running on GCE node
I20260809 00:41:27.032269 28494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:27.032541 28494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:27.033720 28494 hybrid_clock.cc:648] HybridClock initialized: now 1786236087033688 us; error 43 us; skew 500 ppm
I20260809 00:41:27.035238 28494 webserver.cc:460] Webserver started at http://127.7.105.65:38329/ using document root <none> and password file <none>
I20260809 00:41:27.035532 28494 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:27.035616 28494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:27.035784 28494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:27.037369 28494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "8c47e3a4e788453dacd9d6c9960c1328"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "7b4d026253512affc9d9b12faeaa8220"
server_key_iv: "0f44f664d10c7ea262d895230212d162"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.037881 28494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "8c47e3a4e788453dacd9d6c9960c1328"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "7b4d026253512affc9d9b12faeaa8220"
server_key_iv: "0f44f664d10c7ea262d895230212d162"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.039625 28494 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:41:27.040561 28508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.040745 28494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:27.040863 28494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "8c47e3a4e788453dacd9d6c9960c1328"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "7b4d026253512affc9d9b12faeaa8220"
server_key_iv: "0f44f664d10c7ea262d895230212d162"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.040940 28494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:27.059590 28494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:27.059934 28494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:27.060142 28494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:27.060397 28494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:27.060750 28494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:27.060786 28494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.060809 28494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:27.060855 28494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.066680 28494 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:34837
I20260809 00:41:27.066790 28631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:34837 every 8 connection(s)
I20260809 00:41:27.067116 28494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:27.072320 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.067858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:58633 (local address 127.7.105.126:43401)
0809 00:41:27.067943 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.067946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.068199 (+ 253us) server_negotiation.cc:407] Connection header received
0809 00:41:27.068679 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.068680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.068695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.068751 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:27.069417 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.069912 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.070911 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.071144 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.071204 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.072016 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:27.072017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:27.072018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:27.072023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:27.072040 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:27.072045 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:27.072152 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.072234 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.072257 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:41:27.072522 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.067814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43401 (local address 127.7.105.65:58633)
0809 00:41:27.068018 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.068093 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.068577 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.068806 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.068809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.068817 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:27.069342 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.069346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.069998 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.069999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.070820 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.070823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.071232 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.071233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.071369 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.071874 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:27.071874 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:27.071903 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:27.071924 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:27.072163 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:27.072166 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:27.072168 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.072222 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.072304 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260809 00:41:27.073143 28632 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43401
I20260809 00:41:27.073263 28632 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:27.073472 28632 heartbeater.cc:511] Master 127.7.105.126:43401 requested a full tablet report, sending...
I20260809 00:41:27.074057 28432 ts_manager.cc:194] Registered new tserver with Master: 8c47e3a4e788453dacd9d6c9960c1328 (127.7.105.65:34837)
I20260809 00:41:27.074271 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28494
I20260809 00:41:27.074347 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:27.074558 7589 external_mini_cluster.cc:1428] Setting key 51672848797b00d5e3f39b058480a80a
I20260809 00:41:27.074867 28432 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.65:58633
I20260809 00:41:27.075747 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--builtin_ntp_servers=127.7.105.84:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:27.174255 28635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:27.174551 28635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:27.174585 28635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:27.176523 28635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:27.176575 28635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:27.176617 28635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:27.178838 28635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:27.179134 28635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:27.179486 28635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:27.185295 28643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.185304 28641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.185269 28640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:27.189281 28635 server_base.cc:1034] running on GCE node
I20260809 00:41:27.189548 28635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:27.189776 28635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:27.190922 28635 hybrid_clock.cc:648] HybridClock initialized: now 1786236087190910 us; error 29 us; skew 500 ppm
I20260809 00:41:27.192121 28635 webserver.cc:460] Webserver started at http://127.7.105.66:33805/ using document root <none> and password file <none>
I20260809 00:41:27.192387 28635 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:27.192454 28635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:27.192626 28635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:27.193723 28635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "b8d597e588b2dde6c4c9a61cc2c82f6d"
server_key_iv: "ea4c8de2efc1efc87a45a63e91014b15"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.194116 28635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "b8d597e588b2dde6c4c9a61cc2c82f6d"
server_key_iv: "ea4c8de2efc1efc87a45a63e91014b15"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.195499 28635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:27.196372 28650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.196538 28635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:27.196660 28635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "b8d597e588b2dde6c4c9a61cc2c82f6d"
server_key_iv: "ea4c8de2efc1efc87a45a63e91014b15"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.196735 28635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:27.206329 28635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:27.206660 28635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:27.206813 28635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:27.207046 28635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:27.207391 28635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:27.207445 28635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.207487 28635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:27.207513 28635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.213831 28635 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:33061
I20260809 00:41:27.213881 28774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:33061 every 8 connection(s)
I20260809 00:41:27.214223 28635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:41:27.220559 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.214983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:38981 (local address 127.7.105.126:43401)
0809 00:41:27.215047 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.215049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.215259 (+ 210us) server_negotiation.cc:407] Connection header received
0809 00:41:27.216016 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.216017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.216031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.216090 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:27.217161 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.218086 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.219098 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.219307 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.219381 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.220298 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:27.220300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:27.220300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:27.220305 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:27.220319 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:27.220323 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:27.220368 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.220446 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.220479 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:27.220794 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.214933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43401 (local address 127.7.105.66:38981)
0809 00:41:27.215119 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.215160 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.215884 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.216242 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.216246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.216256 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:27.217046 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.217052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.218186 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.218187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.218993 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.218995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.219403 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.219404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.219552 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.220149 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:27.220149 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:27.220178 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:27.220200 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:27.220379 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:27.220384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:27.220387 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.220450 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.220556 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260809 00:41:27.221225 28776 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43401
I20260809 00:41:27.221323 28776 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:27.221470 28776 heartbeater.cc:511] Master 127.7.105.126:43401 requested a full tablet report, sending...
I20260809 00:41:27.221860 28431 ts_manager.cc:194] Registered new tserver with Master: 1f1b95ff2c184c7fba6fcf9b1999cc63 (127.7.105.66:33061)
I20260809 00:41:27.221932 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28635
I20260809 00:41:27.222056 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:27.222244 7589 external_mini_cluster.cc:1428] Setting key 92ffbdcfa298f7cceee38c36e8e20547
I20260809 00:41:27.222344 28431 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.66:38981
I20260809 00:41:27.223228 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--builtin_ntp_servers=127.7.105.84:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:27.309597 28779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:27.309844 28779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:27.309881 28779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:27.311115 28779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:27.311184 28779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:27.311223 28779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:27.312736 28779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:27.313017 28779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:27.313328 28779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:41:27.318805 28779 server_base.cc:1034] running on GCE node
W20260809 00:41:27.318939 28786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.318944 28788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:27.318825 28785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:27.319252 28779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:27.319514 28779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:27.320673 28779 hybrid_clock.cc:648] HybridClock initialized: now 1786236087320668 us; error 36 us; skew 500 ppm
I20260809 00:41:27.321938 28779 webserver.cc:460] Webserver started at http://127.7.105.67:41949/ using document root <none> and password file <none>
I20260809 00:41:27.322216 28779 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:27.322288 28779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:27.322425 28779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:27.323513 28779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "80ac5799df2349b1b7d92bc9377ae319"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "58644da04a2052cab26733f28529941b"
server_key_iv: "af79d1c63571449e89b494ae0d0051bc"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.323886 28779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "80ac5799df2349b1b7d92bc9377ae319"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "58644da04a2052cab26733f28529941b"
server_key_iv: "af79d1c63571449e89b494ae0d0051bc"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.325242 28779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:27.326037 28794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.326211 28779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:41:27.326310 28779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "80ac5799df2349b1b7d92bc9377ae319"
format_stamp: "Formatted at 2026-08-09 00:41:27 on dist-test-slave-1zqn"
server_key: "58644da04a2052cab26733f28529941b"
server_key_iv: "af79d1c63571449e89b494ae0d0051bc"
server_key_version: "encryptionkey@0"
I20260809 00:41:27.326395 28779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:27.345795 28779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:27.346131 28779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:27.346302 28779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:27.346537 28779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:27.346892 28779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:27.346948 28779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.346985 28779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:27.347013 28779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:27.352833 28779 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:42105
I20260809 00:41:27.353065 28917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:42105 every 8 connection(s)
I20260809 00:41:27.353295 28779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:41:27.358886 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.354063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:40949 (local address 127.7.105.126:43401)
0809 00:41:27.354125 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.354127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.354389 (+ 262us) server_negotiation.cc:407] Connection header received
0809 00:41:27.354926 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.354927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.354953 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.355010 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:27.355706 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.356403 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.357415 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.357616 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.357704 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.358568 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:27.358569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:27.358569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:27.358575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:27.358590 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:27.358594 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:27.358670 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.358773 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.358803 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:41:27.359087 28920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.354052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43401 (local address 127.7.105.67:40949)
0809 00:41:27.354257 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.354294 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.354817 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.355080 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.355084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.355091 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:27.355609 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.355613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.356504 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.356505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.357321 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.357323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.357716 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.357717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.357847 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.358415 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:27.358415 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:27.358451 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:27.358474 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:27.358670 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:27.358676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:27.358679 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.358764 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.358867 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260809 00:41:27.359640 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28779
I20260809 00:41:27.359746 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:27.359803 28918 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43401
I20260809 00:41:27.359896 28918 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:27.360078 7589 external_mini_cluster.cc:1428] Setting key 724e678a600a78e0984d19d8af03be31
I20260809 00:41:27.360126 28918 heartbeater.cc:511] Master 127.7.105.126:43401 requested a full tablet report, sending...
I20260809 00:41:27.360586 28432 ts_manager.cc:194] Registered new tserver with Master: 80ac5799df2349b1b7d92bc9377ae319 (127.7.105.67:42105)
I20260809 00:41:27.361115 28432 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.67:40949
I20260809 00:41:27.362030 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:27.366297 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.363482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.7.105.126:43401)
0809 00:41:27.363539 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.363541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.363728 (+ 187us) server_negotiation.cc:407] Connection header received
0809 00:41:27.363751 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.363752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.363761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.363814 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:27.363942 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.364481 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.365188 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.365368 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.365439 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.366054 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:27.366055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:27.366056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:27.366060 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:27.366073 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:27.366077 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:27.366152 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.366232 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.366242 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:41:27.367779 28432 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:27.369452 28432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:27.375824 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.371645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42105 (local address 127.0.0.1:52194)
0809 00:41:27.371803 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.371843 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.371905 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.372280 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.372283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.372285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:27.372359 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.372361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.373500 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.373501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.374757 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.374760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375521 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.375595 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.375619 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.375693 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.375747 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":54,"threads_started":1}
I20260809 00:41:27.375837 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.371810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.7.105.67:42105)
0809 00:41:27.371986 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.371991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.372002 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:41:27.372044 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.372046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.372095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.372183 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:27.372468 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.373398 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.374844 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375432 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375521 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.375614 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.375708 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.375747 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260809 00:41:27.376276 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.371656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34837 (local address 127.0.0.1:51742)
0809 00:41:27.372167 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.372176 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.372213 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.372472 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.372473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.372474 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:27.372504 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.372505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.373859 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.373860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.374896 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.374898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375966 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.376071 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.376076 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.376133 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.376179 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":5,"thread_start_us":77,"threads_started":1}
I20260809 00:41:27.376293 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.371907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.7.105.65:34837)
0809 00:41:27.372309 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.372313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.372323 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:27.372362 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.372364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.372380 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.372453 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:27.372583 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.373769 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.374996 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375735 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375814 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.375916 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.376154 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.376199 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":77,"threads_started":1}
I20260809 00:41:27.376492 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.372007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41488 (local address 127.7.105.66:33061)
0809 00:41:27.372592 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.372596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.372606 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:27.372654 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.372656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.372671 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.372779 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:27.372889 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.373928 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375071 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375774 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375833 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.375916 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.376326 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.376379 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":95,"threads_started":1}
I20260809 00:41:27.376274 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.371649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:33061 (local address 127.0.0.1:41488)
0809 00:41:27.372004 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.372015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.372085 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.372782 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.372783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.372785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:27.372825 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.372826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.374015 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.374016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.374995 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.374997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.375967 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.375968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.376073 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.376077 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.376131 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.376179 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":23,"thread_start_us":72,"threads_started":1}
I20260809 00:41:27.377053 28847 tablet_service.cc:1467] Processing CreateTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 (DEFAULT_TABLE table=test-table [id=661947937129434f938761f442dd3353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:27.377113 28565 tablet_service.cc:1467] Processing CreateTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 (DEFAULT_TABLE table=test-table [id=661947937129434f938761f442dd3353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:27.377458 28847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae10983bfefa4e238c8b0f822a3e39c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:27.377363 28706 tablet_service.cc:1467] Processing CreateTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 (DEFAULT_TABLE table=test-table [id=661947937129434f938761f442dd3353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:27.377489 28565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae10983bfefa4e238c8b0f822a3e39c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:27.377688 28706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae10983bfefa4e238c8b0f822a3e39c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:27.380352 28933 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Bootstrap starting.
I20260809 00:41:27.380990 28935 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Bootstrap starting.
I20260809 00:41:27.381239 28933 tablet_bootstrap.cc:654] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:27.381592 28933 log.cc:824] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:27.381703 28935 tablet_bootstrap.cc:654] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:27.382035 28935 log.cc:824] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:27.382061 28934 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Bootstrap starting.
I20260809 00:41:27.382665 28934 tablet_bootstrap.cc:654] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:27.383018 28934 log.cc:824] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:27.383661 28933 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: No bootstrap required, opened a new log
I20260809 00:41:27.383729 28933 ts_tablet_manager.cc:1397] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:41:27.384418 28934 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: No bootstrap required, opened a new log
I20260809 00:41:27.384479 28934 ts_tablet_manager.cc:1397] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:27.385721 28933 raft_consensus.cc:348] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.385885 28933 raft_consensus.cc:374] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:27.385937 28933 raft_consensus.cc:729] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ac5799df2349b1b7d92bc9377ae319, State: Initialized, Role: FOLLOWER
I20260809 00:41:27.386086 28933 consensus_queue.cc:260] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.386418 28933 ts_tablet_manager.cc:1428] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:41:27.386475 28918 heartbeater.cc:503] Master 127.7.105.126:43401 was elected leader, sending a full tablet report...
I20260809 00:41:27.386777 28934 raft_consensus.cc:348] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.386970 28934 raft_consensus.cc:374] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:27.387013 28934 raft_consensus.cc:729] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c47e3a4e788453dacd9d6c9960c1328, State: Initialized, Role: FOLLOWER
I20260809 00:41:27.387171 28935 tablet_bootstrap.cc:492] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: No bootstrap required, opened a new log
I20260809 00:41:27.387144 28934 consensus_queue.cc:260] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.387251 28935 ts_tablet_manager.cc:1397] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 00:41:27.387394 28934 ts_tablet_manager.cc:1428] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:41:27.387547 28632 heartbeater.cc:503] Master 127.7.105.126:43401 was elected leader, sending a full tablet report...
I20260809 00:41:27.389619 28935 raft_consensus.cc:348] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.389770 28935 raft_consensus.cc:374] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:27.389792 28935 raft_consensus.cc:729] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1b95ff2c184c7fba6fcf9b1999cc63, State: Initialized, Role: FOLLOWER
I20260809 00:41:27.389907 28935 consensus_queue.cc:260] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.390182 28935 ts_tablet_manager.cc:1428] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:41:27.390317 28776 heartbeater.cc:503] Master 127.7.105.126:43401 was elected leader, sending a full tablet report...
W20260809 00:41:27.464787 28777 tablet.cc:2369] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:41:27.567921 28633 tablet.cc:2369] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:27.571902 28941 raft_consensus.cc:482] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:27.572024 28941 raft_consensus.cc:504] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.572484 28941 leader_election.cc:288] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80ac5799df2349b1b7d92bc9377ae319 (127.7.105.67:42105), 8c47e3a4e788453dacd9d6c9960c1328 (127.7.105.65:34837)
I20260809 00:41:27.575460 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.572717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34837 (local address 127.7.105.66:52707)
0809 00:41:27.572759 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.572771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.572815 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.573104 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.573105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.573107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:27.573189 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.573190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.573838 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.573839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.574607 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.574609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.575182 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.575183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.575260 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.575263 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.575331 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.575378 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:41:27.575448 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.572832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:52707 (local address 127.7.105.65:34837)
0809 00:41:27.572886 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.572888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.572896 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:27.572929 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.572930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.573006 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.573094 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:27.573266 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.573747 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.574706 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.575093 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.575181 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.575191 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.575339 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.575378 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:41:27.575628 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.572887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:40553 (local address 127.7.105.67:42105)
0809 00:41:27.572935 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.572937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.573136 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:41:27.573214 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.573215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.573224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.573293 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:27.573468 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.573948 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.574874 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.575233 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.575317 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.575325 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.575487 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.575525 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:27.575794 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.572717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:42105 (local address 127.7.105.66:40553)
0809 00:41:27.573042 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.573052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:27.573153 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:27.573310 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:27.573311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:27.573313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:27.573392 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.573393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.574044 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.574045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.574782 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:27.574784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.575328 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.575329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:27.575399 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.575402 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:27.575478 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:41:27.575645 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":149,"thread_start_us":109,"threads_started":1}
I20260809 00:41:27.575781 28586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae10983bfefa4e238c8b0f822a3e39c7" candidate_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c47e3a4e788453dacd9d6c9960c1328" is_pre_election: true
I20260809 00:41:27.575975 28586 raft_consensus.cc:2454] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1b95ff2c184c7fba6fcf9b1999cc63 in term 0.
I20260809 00:41:27.576164 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae10983bfefa4e238c8b0f822a3e39c7" candidate_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ac5799df2349b1b7d92bc9377ae319" is_pre_election: true
I20260809 00:41:27.576248 28654 leader_election.cc:302] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1b95ff2c184c7fba6fcf9b1999cc63, 8c47e3a4e788453dacd9d6c9960c1328; no voters:
I20260809 00:41:27.576324 28871 raft_consensus.cc:2454] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1b95ff2c184c7fba6fcf9b1999cc63 in term 0.
I20260809 00:41:27.576407 28941 raft_consensus.cc:2781] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:27.576476 28941 raft_consensus.cc:482] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:27.576524 28941 raft_consensus.cc:3037] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:27.577277 28941 raft_consensus.cc:504] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.577425 28941 leader_election.cc:288] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [CANDIDATE]: Term 1 election: Requested vote from peers 80ac5799df2349b1b7d92bc9377ae319 (127.7.105.67:42105), 8c47e3a4e788453dacd9d6c9960c1328 (127.7.105.65:34837)
I20260809 00:41:27.577571 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae10983bfefa4e238c8b0f822a3e39c7" candidate_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ac5799df2349b1b7d92bc9377ae319"
I20260809 00:41:27.577592 28586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae10983bfefa4e238c8b0f822a3e39c7" candidate_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c47e3a4e788453dacd9d6c9960c1328"
I20260809 00:41:27.577696 28586 raft_consensus.cc:3037] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:27.577677 28871 raft_consensus.cc:3037] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:27.578410 28871 raft_consensus.cc:2454] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1b95ff2c184c7fba6fcf9b1999cc63 in term 1.
I20260809 00:41:27.578528 28586 raft_consensus.cc:2454] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1b95ff2c184c7fba6fcf9b1999cc63 in term 1.
I20260809 00:41:27.578583 28653 leader_election.cc:302] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1b95ff2c184c7fba6fcf9b1999cc63, 80ac5799df2349b1b7d92bc9377ae319; no voters:
I20260809 00:41:27.578692 28941 raft_consensus.cc:2781] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:27.578866 28941 raft_consensus.cc:686] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1b95ff2c184c7fba6fcf9b1999cc63, State: Running, Role: LEADER
I20260809 00:41:27.578971 28941 consensus_queue.cc:237] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } }
I20260809 00:41:27.579843 28432 catalog_manager.cc:5495] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f1b95ff2c184c7fba6fcf9b1999cc63 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1b95ff2c184c7fba6fcf9b1999cc63" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 } health_report { overall_health: UNKNOWN } } }
W20260809 00:41:27.603853 28919 tablet.cc:2369] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:27.606827 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.604279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.7.105.66:33061)
0809 00:41:27.604333 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:27.604335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:27.604346 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:41:27.604403 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:27.604404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:27.604432 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:27.604516 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:27.604681 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.605169 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.605983 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:27.606193 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:27.606270 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:27.606427 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:27.606536 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:27.606651 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:27.606704 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 00:41:27.606713 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:27.609043 28871 raft_consensus.cc:1261] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 1 FOLLOWER]: Refusing update from remote peer 1f1b95ff2c184c7fba6fcf9b1999cc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:27.609087 28586 raft_consensus.cc:1261] T ae10983bfefa4e238c8b0f822a3e39c7 P 8c47e3a4e788453dacd9d6c9960c1328 [term 1 FOLLOWER]: Refusing update from remote peer 1f1b95ff2c184c7fba6fcf9b1999cc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:27.609488 28941 consensus_queue.cc:1035] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ac5799df2349b1b7d92bc9377ae319" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:27.609668 28943 consensus_queue.cc:1035] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c47e3a4e788453dacd9d6c9960c1328" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 34837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:27.613197 28946 mvcc.cc:205] Tried to move back new op lower bound from 7316423014843019264 to 7316423014724169728. Current Snapshot: MvccSnapshot[applied={T|T < 7316423014843019264 or (T in {7316423014843019264})}]
I20260809 00:41:27.613675 28948 mvcc.cc:205] Tried to move back new op lower bound from 7316423014843019264 to 7316423014724169728. Current Snapshot: MvccSnapshot[applied={T|T < 7316423014843019264}]
I20260809 00:41:27.613781 28947 mvcc.cc:205] Tried to move back new op lower bound from 7316423014843019264 to 7316423014724169728. Current Snapshot: MvccSnapshot[applied={T|T < 7316423014843019264}]
I20260809 00:41:27.614928 28431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:27.615041 28431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:27.615896 28431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f00c493f636847fe80aadf918b4ac2ab: Sending DeleteTablet for 3 replicas of tablet ae10983bfefa4e238c8b0f822a3e39c7
I20260809 00:41:27.616354 28565 tablet_service.cc:1514] Processing DeleteTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:27 UTC) from {username='slave'} at 127.0.0.1:51742
I20260809 00:41:27.616353 28706 tablet_service.cc:1514] Processing DeleteTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:27 UTC) from {username='slave'} at 127.0.0.1:41488
I20260809 00:41:27.616406 28847 tablet_service.cc:1514] Processing DeleteTablet for tablet ae10983bfefa4e238c8b0f822a3e39c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:27 UTC) from {username='slave'} at 127.0.0.1:52194
I20260809 00:41:27.616688 28958 tablet_replica.cc:330] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: stopping tablet replica
I20260809 00:41:27.616798 28958 raft_consensus.cc:2229] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:41:27.616930 28958 raft_consensus.cc:2258] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:27.616963 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28494
I20260809 00:41:27.617831 28960 tablet_replica.cc:330] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: stopping tablet replica
I20260809 00:41:27.617835 28958 ts_tablet_manager.cc:1905] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:27.617941 28960 raft_consensus.cc:2229] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:27.618064 28960 raft_consensus.cc:2258] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:27.619846 28958 ts_tablet_manager.cc:1918] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:27.619918 28958 log.cc:1196] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/ae10983bfefa4e238c8b0f822a3e39c7
I20260809 00:41:27.620220 28958 ts_tablet_manager.cc:1939] T ae10983bfefa4e238c8b0f822a3e39c7 P 1f1b95ff2c184c7fba6fcf9b1999cc63: Deleting consensus metadata
I20260809 00:41:27.622011 28960 ts_tablet_manager.cc:1905] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:27.623349 28415 catalog_manager.cc:4841] TS 1f1b95ff2c184c7fba6fcf9b1999cc63 (127.7.105.66:33061): tablet ae10983bfefa4e238c8b0f822a3e39c7 (table test-table [id=661947937129434f938761f442dd3353]) successfully deleted
I20260809 00:41:27.624104 28960 ts_tablet_manager.cc:1918] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:27.624183 28960 log.cc:1196] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/ae10983bfefa4e238c8b0f822a3e39c7
I20260809 00:41:27.624457 28960 ts_tablet_manager.cc:1939] T ae10983bfefa4e238c8b0f822a3e39c7 P 80ac5799df2349b1b7d92bc9377ae319: Deleting consensus metadata
I20260809 00:41:27.624922 28417 catalog_manager.cc:4841] TS 80ac5799df2349b1b7d92bc9377ae319 (127.7.105.67:42105): tablet ae10983bfefa4e238c8b0f822a3e39c7 (table test-table [id=661947937129434f938761f442dd3353]) successfully deleted
W20260809 00:41:27.628098 28419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:34837 (error 108)
I20260809 00:41:27.628490 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28635
I20260809 00:41:27.628645 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:27.628485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:34837 (local address 127.0.0.1:51746)
0809 00:41:27.628534 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:27.628605 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 00:41:27.630425 28419 catalog_manager.cc:4584] TS 8c47e3a4e788453dacd9d6c9960c1328 (127.7.105.65:34837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae10983bfefa4e238c8b0f822a3e39c7: Network error: Client connection negotiation failed: client connection to 127.7.105.65:34837: connect: Connection refused (error 111)
I20260809 00:41:27.641064 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28779
I20260809 00:41:27.648173 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28398
2026-08-09T00:41:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (929 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:27 dist-test-slave-1zqn krb5kdc[28965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:27 dist-test-slave-1zqn krb5kdc[28965](info): set up 2 sockets
Aug 09 00:41:27 dist-test-slave-1zqn krb5kdc[28965](info): commencing operation
krb5kdc: starting...
W20260809 00:41:28.755600 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:28 dist-test-slave-1zqn krb5kdc[28965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:28.889400 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38235
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:28.974433 28979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:28.974709 28979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:28.974735 28979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:28.975880 28979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:28.975920 28979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:28.975935 28979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:28.975947 28979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:28.975960 28979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:28.977479 28979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:38235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:28.977766 28979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:28.978001 28979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:28.983053 28986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:28.983047 28985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:28.983340 28979 server_base.cc:1034] running on GCE node
W20260809 00:41:28.983073 28988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:28.983687 28979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:28.984015 28979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:28.985235 28979 hybrid_clock.cc:648] HybridClock initialized: now 1786236088985211 us; error 38 us; skew 500 ppm
Aug 09 00:41:28 dist-test-slave-1zqn krb5kdc[28965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236088, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:28.987926 28979 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:28.988430 28979 webserver.cc:460] Webserver started at http://127.7.105.126:42617/ using document root <none> and password file <none>
I20260809 00:41:28.988680 28979 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:28.988727 28979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:28.988879 28979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:28.989971 28979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "1faf1ffd0a204be0a60f88646134c4fb"
format_stamp: "Formatted at 2026-08-09 00:41:28 on dist-test-slave-1zqn"
server_key: "509565f55fdca525110a31f541ddc109"
server_key_iv: "bb1bfd33540d3f3527a3c89cd28f2766"
server_key_version: "encryptionkey@0"
I20260809 00:41:28.990356 28979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "1faf1ffd0a204be0a60f88646134c4fb"
format_stamp: "Formatted at 2026-08-09 00:41:28 on dist-test-slave-1zqn"
server_key: "509565f55fdca525110a31f541ddc109"
server_key_iv: "bb1bfd33540d3f3527a3c89cd28f2766"
server_key_version: "encryptionkey@0"
I20260809 00:41:28.991757 28979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:41:28.992693 28996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:28.992911 28979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:28.992990 28979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "1faf1ffd0a204be0a60f88646134c4fb"
format_stamp: "Formatted at 2026-08-09 00:41:28 on dist-test-slave-1zqn"
server_key: "509565f55fdca525110a31f541ddc109"
server_key_iv: "bb1bfd33540d3f3527a3c89cd28f2766"
server_key_version: "encryptionkey@0"
I20260809 00:41:28.993039 28979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:29.009931 28979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:29.012382 28979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:29.012562 28979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:29.017079 28979 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:38235
I20260809 00:41:29.017120 29051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:38235 every 8 connection(s)
I20260809 00:41:29.017521 28979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:29.018610 29052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:29.021270 29052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: Bootstrap starting.
I20260809 00:41:29.021955 29052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:29.022285 29052 log.cc:824] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:29.023059 29052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: No bootstrap required, opened a new log
I20260809 00:41:29.025049 29052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } }
I20260809 00:41:29.025202 29052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:29.025228 29052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1faf1ffd0a204be0a60f88646134c4fb, State: Initialized, Role: FOLLOWER
I20260809 00:41:29.025395 29052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } }
I20260809 00:41:29.025432 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 28979
I20260809 00:41:29.025485 29052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:29.025555 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:29.025550 29052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:29.025609 29052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:29.025756 7589 external_mini_cluster.cc:1428] Setting key 7abf4fdf75f68f0f3b201bdf6bf7eb23
I20260809 00:41:29.026477 29052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } }
I20260809 00:41:29.026616 29052 leader_election.cc:302] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1faf1ffd0a204be0a60f88646134c4fb; no voters:
I20260809 00:41:29.026785 29052 leader_election.cc:288] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:29.026814 29060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:29.026978 29060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [term 1 LEADER]: Becoming Leader. State: Replica: 1faf1ffd0a204be0a60f88646134c4fb, State: Running, Role: LEADER
I20260809 00:41:29.027070 29052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:29.027078 29060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } }
I20260809 00:41:29.027429 29060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1faf1ffd0a204be0a60f88646134c4fb. Latest consensus state: current_term: 1 leader_uuid: "1faf1ffd0a204be0a60f88646134c4fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } } }
I20260809 00:41:29.027515 29060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:29.027712 29062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1faf1ffd0a204be0a60f88646134c4fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf1ffd0a204be0a60f88646134c4fb" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 38235 } } }
I20260809 00:41:29.027842 29062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:29.027765 29065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:29.028767 29065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:29.030508 29065 catalog_manager.cc:1347] Generated new cluster ID: a68683259383488f92de215afca6d9b5
I20260809 00:41:29.030570 29065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:29.037268 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.026306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.7.105.126:38235)
0809 00:41:29.026543 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.026555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.026597 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:41:29.026730 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.026741 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.026809 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.028025 (+ 1216us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.028167 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.029933 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.030916 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.031370 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.031494 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.033964 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.033968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.033969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:29.033981 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.035708 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.036100 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.036105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.036105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.036221 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.036438 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.036439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.036440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.036690 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.036876 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.036949 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.037094 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":132,"threads_started":1}
I20260809 00:41:29.058084 29065 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:29.058661 29065 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:29.068159 29065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: Generated new TSK 0
I20260809 00:41:29.068405 29065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
Entry for principal kudu/127.7.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab.
I20260809 00:41:29.097113 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:29.183177 29085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:29.183377 29085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:29.183401 29085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:29.184580 29085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:29.184626 29085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:29.184653 29085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:29.186116 29085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.65.keytab
--principal=kudu/127.7.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:29.186363 29085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:29.186621 29085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:29.191745 29093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.191787 29090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.191895 29091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:29.192265 29085 server_base.cc:1034] running on GCE node
I20260809 00:41:29.192571 29085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:29.192847 29085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:29.194010 29085 hybrid_clock.cc:648] HybridClock initialized: now 1786236089194028 us; error 76 us; skew 500 ppm
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:29.196805 29085 init.cc:377] Logged in from keytab as kudu/127.7.105.65@KRBTEST.COM (short username kudu)
I20260809 00:41:29.197328 29085 webserver.cc:460] Webserver started at http://127.7.105.65:34089/ using document root <none> and password file <none>
I20260809 00:41:29.197593 29085 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:29.197642 29085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:29.197784 29085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:29.198918 29085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "2d2245030c4f438b9b5fb16662c4f0b8"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "43a92e758a4d9a663c965ed324453c3b"
server_key_iv: "63a35c7be02e962ba289111c872fd9d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.199288 29085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "2d2245030c4f438b9b5fb16662c4f0b8"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "43a92e758a4d9a663c965ed324453c3b"
server_key_iv: "63a35c7be02e962ba289111c872fd9d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.200812 29085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:29.201632 29102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.201844 29085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:29.201913 29085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "2d2245030c4f438b9b5fb16662c4f0b8"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "43a92e758a4d9a663c965ed324453c3b"
server_key_iv: "63a35c7be02e962ba289111c872fd9d5"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.202006 29085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:29.229259 29085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:29.231505 29085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:29.231712 29085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:29.231990 29085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:29.232373 29085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:29.232446 29085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.232501 29085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:29.232528 29085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.238961 29085 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:44655
I20260809 00:41:29.239038 29227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:44655 every 8 connection(s)
I20260809 00:41:29.239413 29085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:29.243173 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29085
I20260809 00:41:29.243257 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:29.243462 7589 external_mini_cluster.cc:1428] Setting key 6983045fa067b04c16bc74f90e6f1611
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.65@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:29.250588 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.240312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:54775 (local address 127.7.105.126:38235)
0809 00:41:29.240363 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.240365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.240550 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:41:29.241083 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.241084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.241098 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.241211 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.242358 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.242933 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.244494 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.244732 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.244826 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.247331 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.247333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.247333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:29.247343 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.249059 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.249517 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.249519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.249520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.249588 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.249865 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.249866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.249866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.249997 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.250092 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.250391 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.250487 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:29.250815 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.240243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38235 (local address 127.7.105.65:54775)
0809 00:41:29.240417 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.240460 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.240965 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.241306 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.241319 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.242026 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:29.242235 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.242240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.243131 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.243132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.244395 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.244398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.244840 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.244841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.244989 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.245542 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:29.245551 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:29.247220 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:29.249166 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.249175 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.249177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.249423 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.249666 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.249667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.249668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.249789 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.250075 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:29.250083 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:29.250238 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.250353 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.250541 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260809 00:41:29.251438 29228 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38235
I20260809 00:41:29.251562 29228 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:29.251756 29228 heartbeater.cc:511] Master 127.7.105.126:38235 requested a full tablet report, sending...
I20260809 00:41:29.252481 29013 ts_manager.cc:194] Registered new tserver with Master: 2d2245030c4f438b9b5fb16662c4f0b8 (127.7.105.65:44655)
WARNING: no policy specified for kudu/127.7.105.66@KRBTEST.COM; defaulting to no policy
I20260809 00:41:29.253430 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.65@KRBTEST.COM'} at 127.7.105.65:54775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
Entry for principal kudu/127.7.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab.
I20260809 00:41:29.283468 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:29.367270 29233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:29.367477 29233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:29.367502 29233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:29.368697 29233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:29.368738 29233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:29.368764 29233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:29.370177 29233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.66.keytab
--principal=kudu/127.7.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:29.370363 29233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:29.370537 29233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:29.375694 29242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.375674 29240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:29.375895 29233 server_base.cc:1034] running on GCE node
W20260809 00:41:29.375669 29239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:29.376190 29233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:29.376410 29233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:29.377568 29233 hybrid_clock.cc:648] HybridClock initialized: now 1786236089377549 us; error 32 us; skew 500 ppm
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:29.380366 29233 init.cc:377] Logged in from keytab as kudu/127.7.105.66@KRBTEST.COM (short username kudu)
I20260809 00:41:29.380784 29233 webserver.cc:460] Webserver started at http://127.7.105.66:33303/ using document root <none> and password file <none>
I20260809 00:41:29.380982 29233 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:29.381027 29233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:29.381124 29233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:29.382159 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "2b55f0fbbfe01919ca12f43bf93cf9ad"
server_key_iv: "5df98353c55a7c25fb5491926ad1426d"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.382555 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "2b55f0fbbfe01919ca12f43bf93cf9ad"
server_key_iv: "5df98353c55a7c25fb5491926ad1426d"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.384145 29233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:29.384987 29249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.385226 29233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:29.385354 29233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "2b55f0fbbfe01919ca12f43bf93cf9ad"
server_key_iv: "5df98353c55a7c25fb5491926ad1426d"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.385438 29233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:29.397138 29233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:29.399375 29233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:29.399557 29233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:29.399801 29233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:29.400194 29233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:29.400250 29233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.400293 29233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:29.400321 29233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.406287 29233 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:33147
I20260809 00:41:29.406335 29371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:33147 every 8 connection(s)
I20260809 00:41:29.406677 29233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:41:29.409278 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29233
I20260809 00:41:29.409404 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:29.409622 7589 external_mini_cluster.cc:1428] Setting key 017fdad195ca3333e038de11d316d387
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.66@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:29.418123 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.407424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:57675 (local address 127.7.105.126:38235)
0809 00:41:29.407506 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.407509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.407721 (+ 212us) server_negotiation.cc:407] Connection header received
0809 00:41:29.408328 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.408329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.408341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.408392 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.409498 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.410215 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.411704 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.411899 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.411957 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.414946 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.414948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.414948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:29.414955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.416464 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.417051 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.417052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.417053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.417113 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.417446 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.417447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.417447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.417550 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.417597 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.417976 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.418047 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:29.418457 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.407338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38235 (local address 127.7.105.66:57675)
0809 00:41:29.407561 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.407611 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.408231 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.408453 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.408456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.409049 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:29.409349 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.409357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.410333 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.410333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.411619 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.411622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.411987 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.411988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.412162 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.412803 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:29.412812 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:29.414822 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:29.416552 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.416561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.416572 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.416962 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.417188 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.417190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.417190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.417364 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.417616 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:29.417623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:29.417814 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.417967 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.418180 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260809 00:41:29.419062 29373 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38235
I20260809 00:41:29.419189 29373 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:29.419373 29373 heartbeater.cc:511] Master 127.7.105.126:38235 requested a full tablet report, sending...
I20260809 00:41:29.419809 29013 ts_manager.cc:194] Registered new tserver with Master: a6ae6b08f21742ae9aa7ab8322a9fb08 (127.7.105.66:33147)
I20260809 00:41:29.420297 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.66@KRBTEST.COM'} at 127.7.105.66:57675
WARNING: no policy specified for kudu/127.7.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
Entry for principal kudu/127.7.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab.
I20260809 00:41:29.444106 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:29.536767 29378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:29.537024 29378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:29.537052 29378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:29.538370 29378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:29.538419 29378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:29.538498 29378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:29.540280 29378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.67.keytab
--principal=kudu/127.7.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:29.540555 29378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:29.540817 29378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:29.546463 29384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.546475 29387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.546541 29383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:29.546717 29378 server_base.cc:1034] running on GCE node
I20260809 00:41:29.547016 29378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:29.547273 29378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:29.548446 29378 hybrid_clock.cc:648] HybridClock initialized: now 1786236089548432 us; error 34 us; skew 500 ppm
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:29.551098 29378 init.cc:377] Logged in from keytab as kudu/127.7.105.67@KRBTEST.COM (short username kudu)
I20260809 00:41:29.551537 29378 webserver.cc:460] Webserver started at http://127.7.105.67:42695/ using document root <none> and password file <none>
I20260809 00:41:29.551792 29378 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:29.551880 29378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:29.552052 29378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:29.553613 29378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "ef9cf6afe7824a30b9525a5f113054d0"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "fbcbe17f68d10fee7806edf5150a9939"
server_key_iv: "cfa301356ec2dca6a207b95543269ddf"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.553983 29378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "ef9cf6afe7824a30b9525a5f113054d0"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "fbcbe17f68d10fee7806edf5150a9939"
server_key_iv: "cfa301356ec2dca6a207b95543269ddf"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.555344 29378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:29.556183 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.556360 29378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:29.556458 29378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "ef9cf6afe7824a30b9525a5f113054d0"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "fbcbe17f68d10fee7806edf5150a9939"
server_key_iv: "cfa301356ec2dca6a207b95543269ddf"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.556548 29378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:29.562986 29378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:29.565277 29378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:29.565439 29378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:29.565665 29378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:29.565986 29378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:29.566046 29378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.566088 29378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:29.566143 29378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.572113 29378 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:33543
I20260809 00:41:29.572152 29516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:33543 every 8 connection(s)
I20260809 00:41:29.572474 29378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:41:29 dist-test-slave-1zqn krb5kdc[28965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.67@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:29.580189 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29378
I20260809 00:41:29.580307 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:29.580540 7589 external_mini_cluster.cc:1428] Setting key d1e1cb5542fb25c4522cc7df3f20b313
I20260809 00:41:29.582355 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.573225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:54045 (local address 127.7.105.126:38235)
0809 00:41:29.573308 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.573311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.573509 (+ 198us) server_negotiation.cc:407] Connection header received
0809 00:41:29.574027 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.574029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.574042 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.574107 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.575040 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.575552 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.576575 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.576786 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.576856 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.579083 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.579084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.579085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:29.579092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.580689 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.581258 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.581259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.581260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.581328 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.581622 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.581623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.581623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.581753 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.581857 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.582162 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.582250 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:41:29.582578 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.573165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:38235 (local address 127.7.105.67:54045)
0809 00:41:29.573379 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.573423 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.573889 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.574181 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.574185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.574760 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:29.574950 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.574954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.575633 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.575634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.576488 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.576490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.576868 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.576869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.576966 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.577521 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:29.577529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:29.578991 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:29.580811 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.580819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.580822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.581155 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.581417 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:29.581418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:29.581419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:29.581545 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:29.581850 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:29.581858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:29.582015 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.582136 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.582316 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260809 00:41:29.583096 29517 heartbeater.cc:347] Connected to a master server at 127.7.105.126:38235
I20260809 00:41:29.583189 29517 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:29.583348 29517 heartbeater.cc:511] Master 127.7.105.126:38235 requested a full tablet report, sending...
I20260809 00:41:29.583701 29013 ts_manager.cc:194] Registered new tserver with Master: ef9cf6afe7824a30b9525a5f113054d0 (127.7.105.67:33543)
I20260809 00:41:29.584154 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.105.67@KRBTEST.COM'} at 127.7.105.67:54045
I20260809 00:41:29.592444 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:29.601052 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.593829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.7.105.126:38235)
0809 00:41:29.593920 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.593922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.594121 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:41:29.594252 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.594254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.594268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.594357 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.595023 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.595577 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.596492 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.596707 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.596808 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.598277 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.598279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.598280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:29.598288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.599584 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.600090 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.600091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.600092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.600162 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:29.600386 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:29.600387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:29.600387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:29.600569 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.600694 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.600858 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.600943 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:41:29.602809 29013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:29.604714 29013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:29.611344 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.7.105.65:44655)
0809 00:41:29.607686 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.607691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.607702 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:41:29.607746 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.607748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.607767 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.607845 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:29.608017 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.608856 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.610134 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.610899 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.610971 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611037 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.611213 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.611256 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":121,"threads_started":1}
I20260809 00:41:29.611348 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:44655 (local address 127.0.0.1:45752)
0809 00:41:29.607568 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.607577 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.607634 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.607851 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.607854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.607856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:29.607929 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.607931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.608977 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.608978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.610023 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.610026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.610990 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.610991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.611085 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611100 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.611184 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.611239 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":104,"threads_started":1}
I20260809 00:41:29.611958 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.7.105.67:33543)
0809 00:41:29.607639 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.607645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.607806 (+ 161us) server_negotiation.cc:407] Connection header received
0809 00:41:29.607858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.607861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.607879 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.608042 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:29.608290 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.609525 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.610743 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.611556 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.611637 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611699 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.611810 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.611851 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260809 00:41:29.611963 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:33543 (local address 127.0.0.1:51852)
0809 00:41:29.607702 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.607711 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.607756 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.608150 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.608151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.608153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:29.608199 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.608201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.609642 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.609643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.610635 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.610638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.611638 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.611640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.611706 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611712 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.611796 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.611843 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":102,"threads_started":1}
I20260809 00:41:29.612223 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.7.105.66:33147)
0809 00:41:29.607734 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.607738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.607767 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:41:29.607821 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.607824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.607842 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.607945 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:29.608230 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.609473 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.610769 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.611506 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.611577 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611642 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.612105 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.612152 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":78,"threads_started":1}
I20260809 00:41:29.612483 29155 tablet_service.cc:1467] Processing CreateTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 (DEFAULT_TABLE table=test-table [id=e43d7bb56c354a309f734f071ab21aa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:29.612900 29155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fe58135b024aabb90e3f44e32d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:29.612809 29444 tablet_service.cc:1467] Processing CreateTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 (DEFAULT_TABLE table=test-table [id=e43d7bb56c354a309f734f071ab21aa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:29.613157 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fe58135b024aabb90e3f44e32d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:29.615361 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.607166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:33147 (local address 127.0.0.1:42204)
0809 00:41:29.607352 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.607392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.607460 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.608041 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.608042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.608045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:29.608136 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.608138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.609611 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.609612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.610668 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.610671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.611677 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.611678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.611745 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.611750 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.611811 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.611861 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260809 00:41:29.616232 29303 tablet_service.cc:1467] Processing CreateTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 (DEFAULT_TABLE table=test-table [id=e43d7bb56c354a309f734f071ab21aa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:29.616626 29303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fe58135b024aabb90e3f44e32d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:29.616604 29537 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Bootstrap starting.
I20260809 00:41:29.616938 29538 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Bootstrap starting.
I20260809 00:41:29.617522 29538 tablet_bootstrap.cc:654] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:29.617527 29537 tablet_bootstrap.cc:654] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:29.618029 29538 log.cc:824] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:29.618124 29537 log.cc:824] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:29.618975 29537 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: No bootstrap required, opened a new log
I20260809 00:41:29.618976 29538 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: No bootstrap required, opened a new log
I20260809 00:41:29.619052 29537 ts_tablet_manager.cc:1397] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:41:29.619052 29538 ts_tablet_manager.cc:1397] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:29.620533 29541 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Bootstrap starting.
I20260809 00:41:29.620694 29537 raft_consensus.cc:348] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.620891 29537 raft_consensus.cc:374] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:29.620944 29537 raft_consensus.cc:729] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9cf6afe7824a30b9525a5f113054d0, State: Initialized, Role: FOLLOWER
I20260809 00:41:29.621100 29537 consensus_queue.cc:260] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.621395 29537 ts_tablet_manager.cc:1428] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:41:29.621408 29541 tablet_bootstrap.cc:654] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:29.621294 29538 raft_consensus.cc:348] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.621451 29538 raft_consensus.cc:374] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:29.621517 29538 raft_consensus.cc:729] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2245030c4f438b9b5fb16662c4f0b8, State: Initialized, Role: FOLLOWER
I20260809 00:41:29.621531 29517 heartbeater.cc:503] Master 127.7.105.126:38235 was elected leader, sending a full tablet report...
I20260809 00:41:29.621671 29538 consensus_queue.cc:260] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.621978 29228 heartbeater.cc:503] Master 127.7.105.126:38235 was elected leader, sending a full tablet report...
I20260809 00:41:29.622078 29541 log.cc:824] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:29.622434 29538 ts_tablet_manager.cc:1428] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:41:29.623222 29541 tablet_bootstrap.cc:492] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: No bootstrap required, opened a new log
I20260809 00:41:29.623299 29541 ts_tablet_manager.cc:1397] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:41:29.624871 29541 raft_consensus.cc:348] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.625026 29541 raft_consensus.cc:374] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:29.625102 29541 raft_consensus.cc:729] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ae6b08f21742ae9aa7ab8322a9fb08, State: Initialized, Role: FOLLOWER
I20260809 00:41:29.625237 29541 consensus_queue.cc:260] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.625519 29541 ts_tablet_manager.cc:1428] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:41:29.625547 29373 heartbeater.cc:503] Master 127.7.105.126:38235 was elected leader, sending a full tablet report...
W20260809 00:41:29.657346 29374 tablet.cc:2369] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:41:29.740247 29229 tablet.cc:2369] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:29.759755 29543 raft_consensus.cc:482] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:29.759888 29543 raft_consensus.cc:504] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.760344 29543 leader_election.cc:288] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ae6b08f21742ae9aa7ab8322a9fb08 (127.7.105.66:33147), ef9cf6afe7824a30b9525a5f113054d0 (127.7.105.67:33543)
I20260809 00:41:29.764673 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.760649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:47487 (local address 127.7.105.66:33147)
0809 00:41:29.760700 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.760703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.760803 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:41:29.760859 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.760860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.760872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.760947 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:29.761219 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.761982 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.763609 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.764230 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.764312 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.764321 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.764550 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.764601 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:41:29.764679 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.760651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:55891 (local address 127.7.105.67:33543)
0809 00:41:29.760700 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.760703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.760804 (+ 101us) server_negotiation.cc:407] Connection header received
0809 00:41:29.760859 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.760860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.760872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.761022 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:29.761219 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.762042 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.763609 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.764237 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.764312 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.764321 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.764565 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.764613 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:41:29.764671 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.760554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:33543 (local address 127.7.105.65:55891)
0809 00:41:29.760715 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.760728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.760799 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.761031 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.761032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.761035 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:29.761128 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.761131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.762122 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.762122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.763520 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.763522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.764328 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.764328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.764418 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.764426 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.764527 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.764555 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":98,"thread_start_us":71,"threads_started":1}
I20260809 00:41:29.764671 29546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.760554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:33147 (local address 127.7.105.65:47487)
0809 00:41:29.760717 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.760728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:29.760799 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:29.761047 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:29.761048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:29.761049 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:29.761133 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.761135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.762098 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.762100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.763507 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:29.763510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.764326 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.764328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:29.764419 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.764425 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:41:29.764492 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:41:29.764548 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":95,"thread_start_us":91,"threads_started":1}
I20260809 00:41:29.765103 29324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fe58135b024aabb90e3f44e32d74f5" candidate_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" is_pre_election: true
I20260809 00:41:29.765127 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fe58135b024aabb90e3f44e32d74f5" candidate_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef9cf6afe7824a30b9525a5f113054d0" is_pre_election: true
I20260809 00:41:29.765280 29324 raft_consensus.cc:2454] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2245030c4f438b9b5fb16662c4f0b8 in term 0.
I20260809 00:41:29.765280 29466 raft_consensus.cc:2454] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2245030c4f438b9b5fb16662c4f0b8 in term 0.
I20260809 00:41:29.765574 29103 leader_election.cc:302] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2245030c4f438b9b5fb16662c4f0b8, a6ae6b08f21742ae9aa7ab8322a9fb08; no voters:
I20260809 00:41:29.765755 29543 raft_consensus.cc:2781] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:29.765844 29543 raft_consensus.cc:482] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:29.765882 29543 raft_consensus.cc:3037] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:29.766625 29543 raft_consensus.cc:504] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.766785 29543 leader_election.cc:288] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [CANDIDATE]: Term 1 election: Requested vote from peers a6ae6b08f21742ae9aa7ab8322a9fb08 (127.7.105.66:33147), ef9cf6afe7824a30b9525a5f113054d0 (127.7.105.67:33543)
I20260809 00:41:29.766961 29324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fe58135b024aabb90e3f44e32d74f5" candidate_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08"
I20260809 00:41:29.767086 29324 raft_consensus.cc:3037] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:29.767141 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fe58135b024aabb90e3f44e32d74f5" candidate_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef9cf6afe7824a30b9525a5f113054d0"
I20260809 00:41:29.767230 29466 raft_consensus.cc:3037] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:29.767890 29324 raft_consensus.cc:2454] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2245030c4f438b9b5fb16662c4f0b8 in term 1.
I20260809 00:41:29.768090 29103 leader_election.cc:302] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2245030c4f438b9b5fb16662c4f0b8, a6ae6b08f21742ae9aa7ab8322a9fb08; no voters:
I20260809 00:41:29.768100 29466 raft_consensus.cc:2454] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2245030c4f438b9b5fb16662c4f0b8 in term 1.
I20260809 00:41:29.768239 29543 raft_consensus.cc:2781] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:29.768380 29543 raft_consensus.cc:686] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2245030c4f438b9b5fb16662c4f0b8, State: Running, Role: LEADER
I20260809 00:41:29.768462 29543 consensus_queue.cc:237] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } }
I20260809 00:41:29.769506 29013 catalog_manager.cc:5495] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2245030c4f438b9b5fb16662c4f0b8 (127.7.105.65). New cstate: current_term: 1 leader_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2245030c4f438b9b5fb16662c4f0b8" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 44655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 } health_report { overall_health: UNKNOWN } } }
I20260809 00:41:29.792853 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.790468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.7.105.65:44655)
0809 00:41:29.790533 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.790535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.790544 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:29.790599 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.790600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.790619 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.790672 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:29.790802 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.791273 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.791980 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.792288 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.792345 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.792504 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:29.792603 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:29.792662 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.792727 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.792772 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:41:29.794898 29466 raft_consensus.cc:1261] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 1 FOLLOWER]: Refusing update from remote peer 2d2245030c4f438b9b5fb16662c4f0b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:29.794930 29324 raft_consensus.cc:1261] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 1 FOLLOWER]: Refusing update from remote peer 2d2245030c4f438b9b5fb16662c4f0b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:29.795317 29548 consensus_queue.cc:1035] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef9cf6afe7824a30b9525a5f113054d0" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 33543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:29.795472 29543 consensus_queue.cc:1035] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ae6b08f21742ae9aa7ab8322a9fb08" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 33147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:29.797819 29551 mvcc.cc:205] Tried to move back new op lower bound from 7316423023796875264 to 7316423023692304384. Current Snapshot: MvccSnapshot[applied={T|T < 7316423023796875264}]
I20260809 00:41:29.798418 29553 mvcc.cc:205] Tried to move back new op lower bound from 7316423023796875264 to 7316423023692304384. Current Snapshot: MvccSnapshot[applied={T|T < 7316423023796875264}]
I20260809 00:41:29.798763 29552 mvcc.cc:205] Tried to move back new op lower bound from 7316423023796875264 to 7316423023692304384. Current Snapshot: MvccSnapshot[applied={T|T < 7316423023796875264 or (T in {7316423023796875264})}]
I20260809 00:41:29.800316 29012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:29.800397 29012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:29.801198 29012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faf1ffd0a204be0a60f88646134c4fb: Sending DeleteTablet for 3 replicas of tablet 83fe58135b024aabb90e3f44e32d74f5
I20260809 00:41:29.801519 29155 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:45752
I20260809 00:41:29.801513 29303 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:42204
I20260809 00:41:29.801656 29444 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fe58135b024aabb90e3f44e32d74f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:29 UTC) from {username='kudu', principal='kudu/127.7.105.126@KRBTEST.COM'} at 127.0.0.1:51852
I20260809 00:41:29.801759 29565 tablet_replica.cc:330] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: stopping tablet replica
I20260809 00:41:29.801860 29565 raft_consensus.cc:2229] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:29.801918 29566 tablet_replica.cc:330] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8: stopping tablet replica
I20260809 00:41:29.801970 29565 raft_consensus.cc:2258] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:29.802011 29566 raft_consensus.cc:2229] T 83fe58135b024aabb90e3f44e32d74f5 P 2d2245030c4f438b9b5fb16662c4f0b8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:41:29.802132 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29085
I20260809 00:41:29.804478 29565 ts_tablet_manager.cc:1905] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:29.805996 29567 tablet_replica.cc:330] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: stopping tablet replica
I20260809 00:41:29.806103 29567 raft_consensus.cc:2229] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:29.806201 29567 raft_consensus.cc:2258] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:29.806677 29567 ts_tablet_manager.cc:1905] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:29.806703 29565 ts_tablet_manager.cc:1918] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:29.806758 29565 log.cc:1196] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/83fe58135b024aabb90e3f44e32d74f5
I20260809 00:41:29.806990 29565 ts_tablet_manager.cc:1939] T 83fe58135b024aabb90e3f44e32d74f5 P a6ae6b08f21742ae9aa7ab8322a9fb08: Deleting consensus metadata
I20260809 00:41:29.807428 28997 catalog_manager.cc:4841] TS a6ae6b08f21742ae9aa7ab8322a9fb08 (127.7.105.66:33147): tablet 83fe58135b024aabb90e3f44e32d74f5 (table test-table [id=e43d7bb56c354a309f734f071ab21aa4]) successfully deleted
I20260809 00:41:29.808957 29567 ts_tablet_manager.cc:1918] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:29.809046 29567 log.cc:1196] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/83fe58135b024aabb90e3f44e32d74f5
I20260809 00:41:29.809342 29567 ts_tablet_manager.cc:1939] T 83fe58135b024aabb90e3f44e32d74f5 P ef9cf6afe7824a30b9525a5f113054d0: Deleting consensus metadata
I20260809 00:41:29.809794 28998 catalog_manager.cc:4841] TS ef9cf6afe7824a30b9525a5f113054d0 (127.7.105.67:33543): tablet 83fe58135b024aabb90e3f44e32d74f5 (table test-table [id=e43d7bb56c354a309f734f071ab21aa4]) successfully deleted
W20260809 00:41:29.811398 28999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:44655 (error 108)
I20260809 00:41:29.811952 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.811818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:44655 (local address 127.0.0.1:45758)
0809 00:41:29.811878 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:29.811917 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:44655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:41:29.811981 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29233
W20260809 00:41:29.812136 28999 catalog_manager.cc:4584] TS 2d2245030c4f438b9b5fb16662c4f0b8 (127.7.105.65:44655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83fe58135b024aabb90e3f44e32d74f5: Network error: Client connection negotiation failed: client connection to 127.7.105.65:44655: connect: Connection refused (error 111)
I20260809 00:41:29.819664 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29378
I20260809 00:41:29.825225 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 28979
2026-08-09T00:41:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2178 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T00:41:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:29Z Disabled control of system clock
I20260809 00:41:29.857419 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43253
--webserver_interface=127.7.105.126
--webserver_port=0
--builtin_ntp_servers=127.7.105.84:34525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:29.942082 29573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:29.942304 29573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:29.942325 29573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:29.943585 29573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:29.943629 29573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:29.943641 29573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:29.943655 29573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:29.943666 29573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:29.945220 29573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:43253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:29.945433 29573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:29.945672 29573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:29.950727 29579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.950701 29581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:29.950727 29578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:29.951067 29573 server_base.cc:1034] running on GCE node
I20260809 00:41:29.951287 29573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:29.951598 29573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:29.952746 29573 hybrid_clock.cc:648] HybridClock initialized: now 1786236089952733 us; error 29 us; skew 500 ppm
I20260809 00:41:29.954128 29573 webserver.cc:460] Webserver started at http://127.7.105.126:46659/ using document root <none> and password file <none>
I20260809 00:41:29.954387 29573 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:29.954463 29573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:29.954627 29573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:29.955746 29573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "1d927a2d62234188812db06402fcae38"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "551b086a27111975c84030b2d4eafa7f"
server_key_iv: "9e94c844e916d978788348db72898b69"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.956187 29573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "1d927a2d62234188812db06402fcae38"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "551b086a27111975c84030b2d4eafa7f"
server_key_iv: "9e94c844e916d978788348db72898b69"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.957527 29573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:29.958375 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:29.958580 29573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:29.958702 29573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "1d927a2d62234188812db06402fcae38"
format_stamp: "Formatted at 2026-08-09 00:41:29 on dist-test-slave-1zqn"
server_key: "551b086a27111975c84030b2d4eafa7f"
server_key_iv: "9e94c844e916d978788348db72898b69"
server_key_version: "encryptionkey@0"
I20260809 00:41:29.958788 29573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:29.973959 29573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:29.974291 29573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:29.974447 29573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:29.978618 29573 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:43253
I20260809 00:41:29.978663 29641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:43253 every 8 connection(s)
I20260809 00:41:29.979023 29573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:29.980026 29643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:29.982080 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29573
I20260809 00:41:29.982192 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:29.982383 7589 external_mini_cluster.cc:1428] Setting key 7f3122400d3b335fe26a1a98fec0d055
I20260809 00:41:29.982545 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: Bootstrap starting.
I20260809 00:41:29.983289 29643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:29.983618 29643 log.cc:824] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:29.984437 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: No bootstrap required, opened a new log
I20260809 00:41:29.986747 29643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } }
I20260809 00:41:29.986923 29643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:29.986967 29643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d927a2d62234188812db06402fcae38, State: Initialized, Role: FOLLOWER
I20260809 00:41:29.987102 29643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } }
I20260809 00:41:29.987187 29643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:29.987219 29643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:29.987272 29643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:29.988092 29643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } }
I20260809 00:41:29.988205 29643 leader_election.cc:302] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d927a2d62234188812db06402fcae38; no voters:
I20260809 00:41:29.988379 29643 leader_election.cc:288] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:29.988411 29648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:29.988523 29648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [term 1 LEADER]: Becoming Leader. State: Replica: 1d927a2d62234188812db06402fcae38, State: Running, Role: LEADER
I20260809 00:41:29.988611 29648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } }
I20260809 00:41:29.988683 29643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:29.988860 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:29.982974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.7.105.126:43253)
0809 00:41:29.983181 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:29.983188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:29.983212 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:41:29.983664 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:29.983670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:29.983769 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:29.983936 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:29.984200 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.985580 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.986351 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:29.986995 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:29.987117 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:29.987849 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:29.987853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:29.987854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:29.988092 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:29.988136 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:29.988151 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:29.988223 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:29.988363 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 00:41:29.988526 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260809 00:41:29.988977 29649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d927a2d62234188812db06402fcae38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } } }
I20260809 00:41:29.989070 29649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:29.989126 29650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d927a2d62234188812db06402fcae38. Latest consensus state: current_term: 1 leader_uuid: "1d927a2d62234188812db06402fcae38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d927a2d62234188812db06402fcae38" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 43253 } } }
I20260809 00:41:29.989530 29650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:29.989586 29652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:29.990115 29652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:29.991817 29652 catalog_manager.cc:1347] Generated new cluster ID: 274b38787c294d81aa68df15340146a4
I20260809 00:41:29.991874 29652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:30.003327 29652 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:30.004045 29652 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:30.006801 29652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: Generated new TSK 0
I20260809 00:41:30.006980 29652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:30.009955 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.65:0
--local_ip_for_outbound_sockets=127.7.105.65
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--builtin_ntp_servers=127.7.105.84:34525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:30.097757 29668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:30.098022 29668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:30.098062 29668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:30.099277 29668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:30.099345 29668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:30.099386 29668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.65
I20260809 00:41:30.100857 29668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.105.65
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.65
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:30.101207 29668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:30.101538 29668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:30.107623 29675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:30.107669 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:30.107820 29674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:30.108039 29668 server_base.cc:1034] running on GCE node
I20260809 00:41:30.108304 29668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:30.108566 29668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:30.109737 29668 hybrid_clock.cc:648] HybridClock initialized: now 1786236090109726 us; error 34 us; skew 500 ppm
I20260809 00:41:30.111200 29668 webserver.cc:460] Webserver started at http://127.7.105.65:33325/ using document root <none> and password file <none>
I20260809 00:41:30.111502 29668 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:30.111583 29668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:30.111744 29668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:30.112944 29668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data/instance:
uuid: "e6f79b2568184b21b06e7c6d949a1cc9"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "bb9407d9db110120f2713016cf6bd7a1"
server_key_iv: "18bd4fb148f98ddc2a85ac0bff60241a"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.113341 29668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance:
uuid: "e6f79b2568184b21b06e7c6d949a1cc9"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "bb9407d9db110120f2713016cf6bd7a1"
server_key_iv: "18bd4fb148f98ddc2a85ac0bff60241a"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.114815 29668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:41:30.115787 29684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.115979 29668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:30.116094 29668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal
uuid: "e6f79b2568184b21b06e7c6d949a1cc9"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "bb9407d9db110120f2713016cf6bd7a1"
server_key_iv: "18bd4fb148f98ddc2a85ac0bff60241a"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.116168 29668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:30.139065 29668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:30.139358 29668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:30.139477 29668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:30.139691 29668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:30.140097 29668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:30.140133 29668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.140156 29668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:30.140206 29668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.146351 29668 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.65:46281
I20260809 00:41:30.146462 29808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.65:46281 every 8 connection(s)
I20260809 00:41:30.146817 29668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/data/info.pb
I20260809 00:41:30.152449 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.147668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.65:40537 (local address 127.7.105.126:43253)
0809 00:41:30.147718 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.147721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.147998 (+ 277us) server_negotiation.cc:407] Connection header received
0809 00:41:30.148655 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.148656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.148667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.148727 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:30.149512 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.150020 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.151031 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.151217 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.151271 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.152239 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:30.152240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:30.152240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:30.152244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:30.152256 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:30.152260 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:30.152297 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.152366 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.152387 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:41:30.152705 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.147672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43253 (local address 127.7.105.65:40537)
0809 00:41:30.147863 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.147898 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.148516 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.148809 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.148813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.148821 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:30.149409 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.149415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.150120 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.150121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.150943 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.150946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.151297 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.151298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.151409 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.152038 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:30.152039 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:30.152119 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:30.152154 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:30.152305 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:30.152308 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:30.152310 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.152357 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.152441 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260809 00:41:30.153514 29809 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43253
I20260809 00:41:30.153604 29809 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:30.153756 29809 heartbeater.cc:511] Master 127.7.105.126:43253 requested a full tablet report, sending...
I20260809 00:41:30.154299 29605 ts_manager.cc:194] Registered new tserver with Master: e6f79b2568184b21b06e7c6d949a1cc9 (127.7.105.65:46281)
I20260809 00:41:30.154951 29605 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.65:40537
I20260809 00:41:30.156392 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29668
I20260809 00:41:30.156482 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-0/wal/instance
I20260809 00:41:30.156749 7589 external_mini_cluster.cc:1428] Setting key 91be2df3f13b2b0ad85b1a3ce541fd8b
I20260809 00:41:30.157792 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.66:0
--local_ip_for_outbound_sockets=127.7.105.66
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--builtin_ntp_servers=127.7.105.84:34525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:41:30.247293 29812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:30.247646 29812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:30.247697 29812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:30.249325 29812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:30.249388 29812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:30.249431 29812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.66
I20260809 00:41:30.250795 29812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.105.66
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.66
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:30.251056 29812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:30.251380 29812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:30.256538 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:30.256562 29817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:30.256543 29820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:30.256886 29812 server_base.cc:1034] running on GCE node
I20260809 00:41:30.257155 29812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:30.257400 29812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:30.258540 29812 hybrid_clock.cc:648] HybridClock initialized: now 1786236090258523 us; error 40 us; skew 500 ppm
I20260809 00:41:30.259886 29812 webserver.cc:460] Webserver started at http://127.7.105.66:44025/ using document root <none> and password file <none>
I20260809 00:41:30.260197 29812 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:30.260270 29812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:30.260413 29812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:30.261518 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data/instance:
uuid: "5f1d971158ca452f90c57445fb05bcc1"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "a19a73d406bdc67b10507ec3d9f644ff"
server_key_iv: "f93ab75d87ef596cc594a3e464229ccd"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.261901 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance:
uuid: "5f1d971158ca452f90c57445fb05bcc1"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "a19a73d406bdc67b10507ec3d9f644ff"
server_key_iv: "f93ab75d87ef596cc594a3e464229ccd"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.263307 29812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:30.264122 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.264307 29812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:30.264408 29812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal
uuid: "5f1d971158ca452f90c57445fb05bcc1"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "a19a73d406bdc67b10507ec3d9f644ff"
server_key_iv: "f93ab75d87ef596cc594a3e464229ccd"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.264492 29812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:30.283444 29812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:30.283795 29812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:30.283951 29812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:30.284224 29812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:30.284574 29812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:30.284629 29812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.284672 29812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:30.284698 29812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.290514 29812 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.66:39813
I20260809 00:41:30.290558 29955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.66:39813 every 8 connection(s)
I20260809 00:41:30.290977 29812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/data/info.pb
I20260809 00:41:30.294214 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29812
I20260809 00:41:30.294323 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal/instance
I20260809 00:41:30.294513 7589 external_mini_cluster.cc:1428] Setting key 8bb059fe2c97ec513a7a54e9f3dc6ed5
I20260809 00:41:30.295598 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.105.67:0
--local_ip_for_outbound_sockets=127.7.105.67
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--builtin_ntp_servers=127.7.105.84:34525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:41:30.296972 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.291739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:39517 (local address 127.7.105.126:43253)
0809 00:41:30.291802 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.291804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.292020 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:41:30.292643 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.292644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.292656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.292716 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:30.293424 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.294019 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.295330 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.295663 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.295741 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.296726 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:30.296727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:30.296728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:30.296732 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:30.296748 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:30.296752 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:30.296798 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.296890 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.296911 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:41:30.297258 29958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.291726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43253 (local address 127.7.105.66:39517)
0809 00:41:30.291906 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.291942 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.292503 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.292798 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.292803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.292811 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:30.293339 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.293343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.294125 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.294126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.295246 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.295249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.295820 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.295821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.295989 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.296566 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:30.296566 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:30.296606 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:30.296635 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:30.296814 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:30.296818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:30.296821 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.296886 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.296994 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260809 00:41:30.298074 29956 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43253
I20260809 00:41:30.298182 29956 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:30.298388 29956 heartbeater.cc:511] Master 127.7.105.126:43253 requested a full tablet report, sending...
I20260809 00:41:30.298770 29605 ts_manager.cc:194] Registered new tserver with Master: 5f1d971158ca452f90c57445fb05bcc1 (127.7.105.66:39813)
I20260809 00:41:30.299145 29605 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.66:39517
W20260809 00:41:30.381068 29959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:30.381284 29959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:30.381340 29959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:30.382498 29959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:30.382539 29959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:30.382565 29959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.105.67
I20260809 00:41:30.383850 29959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:34525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.105.67
--webserver_port=0
--tserver_master_addrs=127.7.105.126:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.105.67
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:30.384122 29959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:30.384413 29959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:30.389819 29965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:30.389832 29967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:30.389914 29959 server_base.cc:1034] running on GCE node
W20260809 00:41:30.389858 29964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:30.390383 29959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:30.390683 29959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:30.391886 29959 hybrid_clock.cc:648] HybridClock initialized: now 1786236090391858 us; error 37 us; skew 500 ppm
I20260809 00:41:30.393461 29959 webserver.cc:460] Webserver started at http://127.7.105.67:34655/ using document root <none> and password file <none>
I20260809 00:41:30.393692 29959 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:30.393748 29959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:30.393848 29959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:30.395004 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data/instance:
uuid: "3a8fb1e4a59449a9a063442a2cc5e529"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "82dc199c4683798b5227079177d6ddd1"
server_key_iv: "5b4916eff2a91d051df006b53472d007"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.395313 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance:
uuid: "3a8fb1e4a59449a9a063442a2cc5e529"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "82dc199c4683798b5227079177d6ddd1"
server_key_iv: "5b4916eff2a91d051df006b53472d007"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.396755 29959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 00:41:30.397585 29973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.397747 29959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:30.397833 29959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal
uuid: "3a8fb1e4a59449a9a063442a2cc5e529"
format_stamp: "Formatted at 2026-08-09 00:41:30 on dist-test-slave-1zqn"
server_key: "82dc199c4683798b5227079177d6ddd1"
server_key_iv: "5b4916eff2a91d051df006b53472d007"
server_key_version: "encryptionkey@0"
I20260809 00:41:30.397889 29959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:30.439350 29959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:30.439668 29959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:30.439781 29959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:30.439986 29959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:41:30.440407 29959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:41:30.440444 29959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.440469 29959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:41:30.440521 29959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:30.446530 29959 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.67:37377
I20260809 00:41:30.446575 30095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.67:37377 every 8 connection(s)
I20260809 00:41:30.446928 29959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/data/info.pb
I20260809 00:41:30.452433 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.447746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.67:39369 (local address 127.7.105.126:43253)
0809 00:41:30.447831 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.447833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.448045 (+ 212us) server_negotiation.cc:407] Connection header received
0809 00:41:30.448648 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.448649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.448661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.448729 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:30.449438 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.450000 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.451015 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.451207 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.451271 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.452178 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:30.452180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:30.452180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:30.452184 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:30.452198 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:30.452201 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:30.452248 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.452347 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.452370 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:41:30.452658 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 29959
I20260809 00:41:30.452697 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.447703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:43253 (local address 127.7.105.67:39369)
0809 00:41:30.447906 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.447949 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.448524 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.448787 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.448791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.448800 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:30.449313 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.449317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.450096 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.450097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.450870 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.450872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.451292 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.451292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.451424 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.452012 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:41:30.452013 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:30.452042 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:41:30.452083 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:30.452262 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:30.452266 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:30.452269 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.452323 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.452404 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260809 00:41:30.452773 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal/instance
I20260809 00:41:30.452970 7589 external_mini_cluster.cc:1428] Setting key a8f633b66ca953a1780d2dbb5dfcf7fb
I20260809 00:41:30.453438 30096 heartbeater.cc:347] Connected to a master server at 127.7.105.126:43253
I20260809 00:41:30.453670 30096 heartbeater.cc:464] Registering TS with master...
I20260809 00:41:30.453900 30096 heartbeater.cc:511] Master 127.7.105.126:43253 requested a full tablet report, sending...
I20260809 00:41:30.454299 29605 ts_manager.cc:194] Registered new tserver with Master: 3a8fb1e4a59449a9a063442a2cc5e529 (127.7.105.67:37377)
I20260809 00:41:30.454650 29605 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.105.67:39369
I20260809 00:41:30.465230 7589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:41:30.469905 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.466761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.7.105.126:43253)
0809 00:41:30.466826 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.466829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.466923 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:41:30.467013 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.467014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.467024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.467103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:30.467320 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.467806 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.468635 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.468817 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.468884 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.469656 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:30.469657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:30.469657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:30.469661 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:30.469671 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:30.469674 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:30.469715 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.469823 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.469842 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:41:30.471589 29605 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:41:30.473454 29605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:41:30.480088 30105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.475940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.66:39813 (local address 127.0.0.1:46398)
0809 00:41:30.476454 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.476500 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.476648 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.476964 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.476967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.476970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:30.477041 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.477043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.477906 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.477907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.479010 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.479012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.479766 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.479767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.479845 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.479859 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.479933 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.479980 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":87,"threads_started":1}
I20260809 00:41:30.480113 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.476033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.7.105.66:39813)
0809 00:41:30.476228 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.476233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.476573 (+ 340us) server_negotiation.cc:407] Connection header received
0809 00:41:30.476773 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.476775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.476788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.476862 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:30.477101 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.477816 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.479096 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.479672 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.479757 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.479850 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.479957 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.480001 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260809 00:41:30.481010 29885 tablet_service.cc:1467] Processing CreateTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 (DEFAULT_TABLE table=test-table [id=2ecd84aad10f45f6a23e439c553466f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:30.481462 29885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b17b08ab2dc4da9b279ff45d14763c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:30.481667 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.476284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.7.105.65:46281)
0809 00:41:30.476524 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.476529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.476634 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:41:30.476749 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.476751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.476777 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.476851 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:30.477135 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.478319 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.480517 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.481295 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.481366 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.481433 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.481540 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.481584 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":97,"threads_started":1}
I20260809 00:41:30.481839 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.476321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.7.105.67:37377)
0809 00:41:30.476522 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.476527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.476832 (+ 305us) server_negotiation.cc:407] Connection header received
0809 00:41:30.476879 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.476881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.476895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.476973 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:30.477136 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.478301 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.479468 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.480429 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.480521 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.480611 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.481703 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.481750 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260809 00:41:30.481909 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.476031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:46281 (local address 127.0.0.1:59912)
0809 00:41:30.476574 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.476585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.476650 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.476963 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.476968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.476974 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:30.477046 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.477049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.478408 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.478409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.480420 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.480423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.481378 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.481380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.481443 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.481447 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.481604 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.481657 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":893,"thread_start_us":85,"threads_started":1}
I20260809 00:41:30.482100 30108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.476092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:37377 (local address 127.0.0.1:46566)
0809 00:41:30.476752 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.476761 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.476813 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.476977 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.476978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.476979 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:30.477046 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.477048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.478449 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.478450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.479376 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.479378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.481465 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.481466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.481549 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.481552 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.481605 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.481646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260809 00:41:30.482612 29738 tablet_service.cc:1467] Processing CreateTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 (DEFAULT_TABLE table=test-table [id=2ecd84aad10f45f6a23e439c553466f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:30.482934 29738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b17b08ab2dc4da9b279ff45d14763c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:30.482805 30025 tablet_service.cc:1467] Processing CreateTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 (DEFAULT_TABLE table=test-table [id=2ecd84aad10f45f6a23e439c553466f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:41:30.483188 30025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b17b08ab2dc4da9b279ff45d14763c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:30.485226 30111 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Bootstrap starting.
I20260809 00:41:30.485672 30113 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Bootstrap starting.
I20260809 00:41:30.486027 30111 tablet_bootstrap.cc:654] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:30.486343 30113 tablet_bootstrap.cc:654] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:30.486371 30111 log.cc:824] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:30.486722 30113 log.cc:824] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:30.486722 30114 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Bootstrap starting.
I20260809 00:41:30.487289 30114 tablet_bootstrap.cc:654] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:30.487514 30113 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: No bootstrap required, opened a new log
I20260809 00:41:30.487571 30113 ts_tablet_manager.cc:1397] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:30.487600 30114 log.cc:824] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:30.488350 30114 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: No bootstrap required, opened a new log
I20260809 00:41:30.488360 30111 tablet_bootstrap.cc:492] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: No bootstrap required, opened a new log
I20260809 00:41:30.488426 30111 ts_tablet_manager.cc:1397] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:41:30.488430 30114 ts_tablet_manager.cc:1397] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:41:30.489933 30113 raft_consensus.cc:348] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.490134 30113 raft_consensus.cc:374] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:30.490181 30113 raft_consensus.cc:729] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f79b2568184b21b06e7c6d949a1cc9, State: Initialized, Role: FOLLOWER
I20260809 00:41:30.490319 30113 consensus_queue.cc:260] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.490612 30113 ts_tablet_manager.cc:1428] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:41:30.490638 29809 heartbeater.cc:503] Master 127.7.105.126:43253 was elected leader, sending a full tablet report...
I20260809 00:41:30.491509 30111 raft_consensus.cc:348] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.491679 30111 raft_consensus.cc:374] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:30.491717 30111 raft_consensus.cc:729] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f1d971158ca452f90c57445fb05bcc1, State: Initialized, Role: FOLLOWER
I20260809 00:41:30.491621 30114 raft_consensus.cc:348] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.491762 30114 raft_consensus.cc:374] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:30.491814 30114 raft_consensus.cc:729] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a8fb1e4a59449a9a063442a2cc5e529, State: Initialized, Role: FOLLOWER
I20260809 00:41:30.491835 30111 consensus_queue.cc:260] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.491918 30114 consensus_queue.cc:260] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.492134 30111 ts_tablet_manager.cc:1428] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 00:41:30.492156 30114 ts_tablet_manager.cc:1428] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260809 00:41:30.492185 29956 heartbeater.cc:503] Master 127.7.105.126:43253 was elected leader, sending a full tablet report...
I20260809 00:41:30.492409 30096 heartbeater.cc:503] Master 127.7.105.126:43253 was elected leader, sending a full tablet report...
W20260809 00:41:30.541543 29957 tablet.cc:2369] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:41:30.647643 29810 tablet.cc:2369] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:41:30.697541 30098 tablet.cc:2369] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:41:30.712826 30119 raft_consensus.cc:482] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:41:30.713088 30119 raft_consensus.cc:504] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.713577 30119 leader_election.cc:288] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a8fb1e4a59449a9a063442a2cc5e529 (127.7.105.67:37377), e6f79b2568184b21b06e7c6d949a1cc9 (127.7.105.65:46281)
I20260809 00:41:30.717726 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.713885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:48877 (local address 127.7.105.67:37377)
0809 00:41:30.713943 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.713946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.713961 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:41:30.714009 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.714009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.714047 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.714143 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:30.714418 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.715169 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.716796 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.717350 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.717460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.717468 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.717606 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.717647 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:41:30.717726 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.713908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.105.66:42489 (local address 127.7.105.65:46281)
0809 00:41:30.713961 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.713964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.714053 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:41:30.714088 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.714089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.714103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.714175 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:41:30.714417 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.715232 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.716797 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.717350 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.717434 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.717444 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.717607 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.717647 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:41:30.717777 29958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.713777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.67:37377 (local address 127.7.105.66:48877)
0809 00:41:30.713855 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.713869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.713917 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.714185 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.714186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.714188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:30.714296 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.714297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.715296 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.715298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.716681 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.716684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.717441 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.717442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.717521 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.717526 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.717584 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.717685 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":155}
I20260809 00:41:30.717784 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.713777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:46281 (local address 127.7.105.66:42489)
0809 00:41:30.713967 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.713975 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:30.714032 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:30.714185 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:30.714186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:30.714188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:41:30.714292 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.714294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.715307 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.715308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.716681 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:30.716684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.717441 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.717442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:30.717521 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.717526 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:41:30.717584 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:41:30.717680 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":68,"thread_start_us":112,"threads_started":1}
I20260809 00:41:30.718179 30046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b17b08ab2dc4da9b279ff45d14763c3" candidate_uuid: "5f1d971158ca452f90c57445fb05bcc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" is_pre_election: true
I20260809 00:41:30.718216 29760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b17b08ab2dc4da9b279ff45d14763c3" candidate_uuid: "5f1d971158ca452f90c57445fb05bcc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" is_pre_election: true
I20260809 00:41:30.718379 30046 raft_consensus.cc:2454] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f1d971158ca452f90c57445fb05bcc1 in term 0.
I20260809 00:41:30.718389 29760 raft_consensus.cc:2454] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f1d971158ca452f90c57445fb05bcc1 in term 0.
I20260809 00:41:30.718663 29831 leader_election.cc:302] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f1d971158ca452f90c57445fb05bcc1, e6f79b2568184b21b06e7c6d949a1cc9; no voters:
I20260809 00:41:30.718796 30119 raft_consensus.cc:2781] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:41:30.718852 30119 raft_consensus.cc:482] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:41:30.718887 30119 raft_consensus.cc:3037] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:30.719651 30119 raft_consensus.cc:504] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.719803 30119 leader_election.cc:288] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [CANDIDATE]: Term 1 election: Requested vote from peers 3a8fb1e4a59449a9a063442a2cc5e529 (127.7.105.67:37377), e6f79b2568184b21b06e7c6d949a1cc9 (127.7.105.65:46281)
I20260809 00:41:30.720170 29760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b17b08ab2dc4da9b279ff45d14763c3" candidate_uuid: "5f1d971158ca452f90c57445fb05bcc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f79b2568184b21b06e7c6d949a1cc9"
I20260809 00:41:30.720204 30046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b17b08ab2dc4da9b279ff45d14763c3" candidate_uuid: "5f1d971158ca452f90c57445fb05bcc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a8fb1e4a59449a9a063442a2cc5e529"
I20260809 00:41:30.720253 29760 raft_consensus.cc:3037] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:30.720348 30046 raft_consensus.cc:3037] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:30.721084 29760 raft_consensus.cc:2454] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f1d971158ca452f90c57445fb05bcc1 in term 1.
I20260809 00:41:30.721246 29831 leader_election.cc:302] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f1d971158ca452f90c57445fb05bcc1, e6f79b2568184b21b06e7c6d949a1cc9; no voters:
I20260809 00:41:30.721330 30119 raft_consensus.cc:2781] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:30.721426 30046 raft_consensus.cc:2454] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f1d971158ca452f90c57445fb05bcc1 in term 1.
I20260809 00:41:30.721478 30119 raft_consensus.cc:686] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 1 LEADER]: Becoming Leader. State: Replica: 5f1d971158ca452f90c57445fb05bcc1, State: Running, Role: LEADER
I20260809 00:41:30.721701 30119 consensus_queue.cc:237] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } }
I20260809 00:41:30.722643 29605 catalog_manager.cc:5495] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f1d971158ca452f90c57445fb05bcc1 (127.7.105.66). New cstate: current_term: 1 leader_uuid: "5f1d971158ca452f90c57445fb05bcc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f1d971158ca452f90c57445fb05bcc1" member_type: VOTER last_known_addr { host: "127.7.105.66" port: 39813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 } health_report { overall_health: UNKNOWN } } }
I20260809 00:41:30.767771 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.764842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.7.105.66:39813)
0809 00:41:30.764923 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:30.764925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:30.764935 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:30.765050 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:30.765051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:30.765072 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:30.765138 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:30.765322 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.765819 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.766912 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:30.767227 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:30.767286 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:30.767480 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:30.767585 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:30.767647 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:30.767686 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:41:30.767707 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:41:30.769719 29760 raft_consensus.cc:1261] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 1 FOLLOWER]: Refusing update from remote peer 5f1d971158ca452f90c57445fb05bcc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:30.769773 30046 raft_consensus.cc:1261] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 1 FOLLOWER]: Refusing update from remote peer 5f1d971158ca452f90c57445fb05bcc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:30.770179 30122 consensus_queue.cc:1035] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f79b2568184b21b06e7c6d949a1cc9" member_type: VOTER last_known_addr { host: "127.7.105.65" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:30.770351 30119 consensus_queue.cc:1035] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a8fb1e4a59449a9a063442a2cc5e529" member_type: VOTER last_known_addr { host: "127.7.105.67" port: 37377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:30.773236 30127 mvcc.cc:205] Tried to move back new op lower bound from 7316423027790012416 to 7316423027596763136. Current Snapshot: MvccSnapshot[applied={T|T < 7316423027790012416}]
I20260809 00:41:30.775216 29605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:30.775301 29605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:41:30.776157 29605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d927a2d62234188812db06402fcae38: Sending DeleteTablet for 3 replicas of tablet 2b17b08ab2dc4da9b279ff45d14763c3
I20260809 00:41:30.776495 30025 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:30 UTC) from {username='slave'} at 127.0.0.1:46566
I20260809 00:41:30.776494 29885 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:30 UTC) from {username='slave'} at 127.0.0.1:46398
I20260809 00:41:30.776736 29738 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b17b08ab2dc4da9b279ff45d14763c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:41:30 UTC) from {username='slave'} at 127.0.0.1:59912
I20260809 00:41:30.776769 30138 tablet_replica.cc:330] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: stopping tablet replica
I20260809 00:41:30.776873 30138 raft_consensus.cc:2229] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:30.776952 30140 tablet_replica.cc:330] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: stopping tablet replica
I20260809 00:41:30.776970 30138 raft_consensus.cc:2258] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:30.777047 30140 raft_consensus.cc:2229] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:41:30.777166 30140 raft_consensus.cc:2258] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:30.777458 30138 ts_tablet_manager.cc:1905] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:30.777469 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29668
I20260809 00:41:30.777612 30140 ts_tablet_manager.cc:1905] T 2b17b08ab2dc4da9b279ff45d14763c3 P e6f79b2568184b21b06e7c6d949a1cc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:30.779186 30139 tablet_replica.cc:330] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: stopping tablet replica
I20260809 00:41:30.779330 30138 ts_tablet_manager.cc:1918] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:30.779392 30138 log.cc:1196] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-2/wal/wals/2b17b08ab2dc4da9b279ff45d14763c3
I20260809 00:41:30.779678 30138 ts_tablet_manager.cc:1939] T 2b17b08ab2dc4da9b279ff45d14763c3 P 3a8fb1e4a59449a9a063442a2cc5e529: Deleting consensus metadata
I20260809 00:41:30.780040 30139 raft_consensus.cc:2229] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:41:30.780123 29588 catalog_manager.cc:4841] TS 3a8fb1e4a59449a9a063442a2cc5e529 (127.7.105.67:37377): tablet 2b17b08ab2dc4da9b279ff45d14763c3 (table test-table [id=2ecd84aad10f45f6a23e439c553466f2]) successfully deleted
I20260809 00:41:30.780175 30139 raft_consensus.cc:2258] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:41:30.780532 30139 ts_tablet_manager.cc:1905] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:41:30.782094 30139 ts_tablet_manager.cc:1918] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:41:30.782168 30139 log.cc:1196] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Deleting WAL directory at /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235843038573-7589-0/minicluster-data/ts-1/wal/wals/2b17b08ab2dc4da9b279ff45d14763c3
I20260809 00:41:30.782420 30139 ts_tablet_manager.cc:1939] T 2b17b08ab2dc4da9b279ff45d14763c3 P 5f1d971158ca452f90c57445fb05bcc1: Deleting consensus metadata
I20260809 00:41:30.782833 29592 catalog_manager.cc:4841] TS 5f1d971158ca452f90c57445fb05bcc1 (127.7.105.66:39813): tablet 2b17b08ab2dc4da9b279ff45d14763c3 (table test-table [id=2ecd84aad10f45f6a23e439c553466f2]) successfully deleted
W20260809 00:41:30.787431 29590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.105.65:46281 (error 108)
I20260809 00:41:30.787798 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29812
I20260809 00:41:30.787925 30105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:30.787788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.65:46281 (local address 127.0.0.1:59916)
0809 00:41:30.787835 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:30.787883 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.65:46281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 00:41:30.790220 29590 catalog_manager.cc:4584] TS e6f79b2568184b21b06e7c6d949a1cc9 (127.7.105.65:46281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b17b08ab2dc4da9b279ff45d14763c3: Network error: Client connection negotiation failed: client connection to 127.7.105.65:46281: connect: Connection refused (error 111)
I20260809 00:41:30.794180 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29959
I20260809 00:41:30.800529 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 29573
2026-08-09T00:41:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (973 ms)
[----------] 6 tests from EncryptionPolicyTest (9276 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:30 dist-test-slave-1zqn krb5kdc[30144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:30 dist-test-slave-1zqn krb5kdc[30144](info): set up 2 sockets
Aug 09 00:41:30 dist-test-slave-1zqn krb5kdc[30144](info): commencing operation
krb5kdc: starting...
W20260809 00:41:31.970366 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:32.099603 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:32.183950 30159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:32.184226 30159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:32.184268 30159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:32.185420 30159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:32.185494 30159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:32.185518 30159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:32.185539 30159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:32.185570 30159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:32.187088 30159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:32.187381 30159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:32.187631 30159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:32.192895 30168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:32.192879 30165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:32.193008 30166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:32.192931 30159 server_base.cc:1034] running on GCE node
I20260809 00:41:32.193602 30159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:32.193863 30159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:32.195061 30159 hybrid_clock.cc:648] HybridClock initialized: now 1786236092195043 us; error 33 us; skew 500 ppm
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:32.203192 30159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:32.203662 30159 webserver.cc:460] Webserver started at http://127.0.0.1:45663/ using document root <none> and password file <none>
I20260809 00:41:32.203891 30159 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:32.203934 30159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:32.204043 30159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:32.205204 30159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "9a87ad24c8f34afcb9a0442a8980c0af"
format_stamp: "Formatted at 2026-08-09 00:41:32 on dist-test-slave-1zqn"
server_key: "295b0a9ee83e62797a1685d4e50c8f86"
server_key_iv: "8a1079ab9b54d9216a6b39bf125269c9"
server_key_version: "encryptionkey@0"
I20260809 00:41:32.205524 30159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "9a87ad24c8f34afcb9a0442a8980c0af"
format_stamp: "Formatted at 2026-08-09 00:41:32 on dist-test-slave-1zqn"
server_key: "295b0a9ee83e62797a1685d4e50c8f86"
server_key_iv: "8a1079ab9b54d9216a6b39bf125269c9"
server_key_version: "encryptionkey@0"
I20260809 00:41:32.206861 30159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:41:32.207693 30175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:32.208117 30159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:32.208251 30159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "9a87ad24c8f34afcb9a0442a8980c0af"
format_stamp: "Formatted at 2026-08-09 00:41:32 on dist-test-slave-1zqn"
server_key: "295b0a9ee83e62797a1685d4e50c8f86"
server_key_iv: "8a1079ab9b54d9216a6b39bf125269c9"
server_key_version: "encryptionkey@0"
I20260809 00:41:32.208336 30159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:32.225153 30159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:32.227617 30159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:32.227787 30159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:32.232111 30159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36795
I20260809 00:41:32.232178 30231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36795 every 8 connection(s)
I20260809 00:41:32.232551 30159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:32.233623 30232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:32.235438 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30159
I20260809 00:41:32.235584 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:32.235812 7589 external_mini_cluster.cc:1428] Setting key 037120b4c2144853503caffecf26a5ac
I20260809 00:41:32.237058 30232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af: Bootstrap starting.
I20260809 00:41:32.237728 30232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:32.238101 30232 log.cc:824] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:32.238961 30232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af: No bootstrap required, opened a new log
I20260809 00:41:32.241293 30232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } }
I20260809 00:41:32.241508 30232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:32.241571 30232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a87ad24c8f34afcb9a0442a8980c0af, State: Initialized, Role: FOLLOWER
I20260809 00:41:32.241772 30232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } }
I20260809 00:41:32.241881 30232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:32.241938 30232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:32.242020 30232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:32.242589 30232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } }
I20260809 00:41:32.242719 30232 leader_election.cc:302] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a87ad24c8f34afcb9a0442a8980c0af; no voters:
I20260809 00:41:32.242889 30232 leader_election.cc:288] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:32.242933 30237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:32.243135 30237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [term 1 LEADER]: Becoming Leader. State: Replica: 9a87ad24c8f34afcb9a0442a8980c0af, State: Running, Role: LEADER
I20260809 00:41:32.243225 30232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:32.243283 30237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } }
I20260809 00:41:32.243687 30237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } } }
I20260809 00:41:32.243860 30237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:32.244081 30237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a87ad24c8f34afcb9a0442a8980c0af. Latest consensus state: current_term: 1 leader_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a87ad24c8f34afcb9a0442a8980c0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36795 } } }
I20260809 00:41:32.244210 30237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:32.244390 30246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:32.244935 30246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:41:32.250221 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:32.236481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41013 (local address 127.0.0.1:36795)
0809 00:41:32.236733 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:32.236743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:32.236781 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:41:32.237363 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:32.237370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:32.237429 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:32.237548 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:32.238405 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:32.239798 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:32.240697 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:32.241402 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:32.241525 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:32.246069 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:32.246074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:32.246075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:32.246088 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:32.248292 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:32.248805 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:32.248809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:32.248809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:32.248914 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:32.249147 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:32.249148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:32.249149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:32.249390 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:32.249547 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:32.249740 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 00:41:32.249956 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":66,"threads_started":1}
I20260809 00:41:32.250387 30246 catalog_manager.cc:1347] Generated new cluster ID: cf3bb0ccfb1540cd8b857c41d0677986
I20260809 00:41:32.250468 30246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:32.251255 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:32.262575 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:32.253014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50619 (local address 127.0.0.1:36795)
0809 00:41:32.253054 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:32.253056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:32.256251 (+ 3195us) server_negotiation.cc:407] Connection header received
0809 00:41:32.256325 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:32.256326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:32.256336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:32.256382 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:32.256934 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:32.257478 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:32.258210 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:32.258477 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:32.258565 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:32.259799 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:32.259800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:32.259800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:32.259808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:32.261336 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:32.261725 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:32.261727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:32.261727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:32.261787 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:32.262020 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:32.262021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:32.262022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:32.262153 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:32.262270 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:32.262380 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:41:32.262454 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:32.267688 30246 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:32.268440 30246 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:32.276142 30246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a87ad24c8f34afcb9a0442a8980c0af: Generated new TSK 0
I20260809 00:41:32.276348 30246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:32.280493 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30159
W20260809 00:41:32.289090 30142 connection.cc:535] client connection to 127.0.0.1:36795 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36795): Connection reset by peer (error 104)
2026-08-09T00:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30266](info): set up 2 sockets
Aug 09 00:41:32 dist-test-slave-1zqn krb5kdc[30266](info): commencing operation
krb5kdc: starting...
W20260809 00:41:33.423494 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:33 dist-test-slave-1zqn krb5kdc[30266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:33.554831 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:33.637857 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:33.638078 30279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:33.638101 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:33.639281 30279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:33.639325 30279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:33.639339 30279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:33.639353 30279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:33.639389 30279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:33.641304 30279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:33.641570 30279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:33.641817 30279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:33.646904 30284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:33.646895 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:33.646895 30287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:33.646924 30279 server_base.cc:1034] running on GCE node
I20260809 00:41:33.647436 30279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:33.647733 30279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:33.648900 30279 hybrid_clock.cc:648] HybridClock initialized: now 1786236093648888 us; error 47 us; skew 500 ppm
Aug 09 00:41:33 dist-test-slave-1zqn krb5kdc[30266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236093, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:33.651667 30279 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:33.652233 30279 webserver.cc:460] Webserver started at http://127.0.0.1:40873/ using document root <none> and password file <none>
I20260809 00:41:33.652486 30279 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:33.652565 30279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:33.652724 30279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:33.653906 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "a404e61585604963acd0aa85285f9a63"
format_stamp: "Formatted at 2026-08-09 00:41:33 on dist-test-slave-1zqn"
server_key: "d4d1c604efa00f3a60fc0c56cfc0a012"
server_key_iv: "21bbb21ca1861bce62735fe3c5fb6c88"
server_key_version: "encryptionkey@0"
I20260809 00:41:33.654284 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "a404e61585604963acd0aa85285f9a63"
format_stamp: "Formatted at 2026-08-09 00:41:33 on dist-test-slave-1zqn"
server_key: "d4d1c604efa00f3a60fc0c56cfc0a012"
server_key_iv: "21bbb21ca1861bce62735fe3c5fb6c88"
server_key_version: "encryptionkey@0"
I20260809 00:41:33.655720 30279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:33.656643 30295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:33.656855 30279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:41:33.656968 30279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "a404e61585604963acd0aa85285f9a63"
format_stamp: "Formatted at 2026-08-09 00:41:33 on dist-test-slave-1zqn"
server_key: "d4d1c604efa00f3a60fc0c56cfc0a012"
server_key_iv: "21bbb21ca1861bce62735fe3c5fb6c88"
server_key_version: "encryptionkey@0"
I20260809 00:41:33.657083 30279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:33.675550 30279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:33.677748 30279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:33.677927 30279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:33.682446 30279 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36661
I20260809 00:41:33.682495 30350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36661 every 8 connection(s)
I20260809 00:41:33.682888 30279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:33.683945 30351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:33.686344 30351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63: Bootstrap starting.
I20260809 00:41:33.686882 30351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:33.687144 30351 log.cc:824] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:33.687815 30351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63: No bootstrap required, opened a new log
I20260809 00:41:33.689391 30351 raft_consensus.cc:348] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260809 00:41:33.689530 30351 raft_consensus.cc:374] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:33.689553 30351 raft_consensus.cc:729] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a404e61585604963acd0aa85285f9a63, State: Initialized, Role: FOLLOWER
I20260809 00:41:33.689697 30351 consensus_queue.cc:260] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260809 00:41:33.689785 30351 raft_consensus.cc:388] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:33.689810 30351 raft_consensus.cc:482] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:33.689837 30351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:33.690138 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30279
I20260809 00:41:33.690240 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:33.690423 30351 raft_consensus.cc:504] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260809 00:41:33.690459 7589 external_mini_cluster.cc:1428] Setting key fefbec2ec58a25104ad6267ce5ea8a38
I20260809 00:41:33.690544 30351 leader_election.cc:302] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a404e61585604963acd0aa85285f9a63; no voters:
I20260809 00:41:33.690740 30351 leader_election.cc:288] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:33.690763 30355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:33.690894 30355 raft_consensus.cc:686] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [term 1 LEADER]: Becoming Leader. State: Replica: a404e61585604963acd0aa85285f9a63, State: Running, Role: LEADER
I20260809 00:41:33.691021 30351 sys_catalog.cc:564] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:33.691033 30355 consensus_queue.cc:237] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260809 00:41:33.691798 30356 sys_catalog.cc:455] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a404e61585604963acd0aa85285f9a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260809 00:41:33.691890 30356 sys_catalog.cc:458] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:33.692122 30357 sys_catalog.cc:455] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a404e61585604963acd0aa85285f9a63. Latest consensus state: current_term: 1 leader_uuid: "a404e61585604963acd0aa85285f9a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a404e61585604963acd0aa85285f9a63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260809 00:41:33.692262 30357 sys_catalog.cc:458] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:33.692414 30368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:33.692981 30368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:33.694657 30368 catalog_manager.cc:1347] Generated new cluster ID: c6429ef259214a26914158747c2383e5
I20260809 00:41:33.694715 30368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:33.696938 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:33.691157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44485 (local address 127.0.0.1:36661)
0809 00:41:33.691346 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:33.691357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:33.691387 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:41:33.691498 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:33.691504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:33.691579 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:33.691981 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:33.692138 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:33.693465 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:33.694604 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:33.695208 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:33.695630 (+ 422us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:33.696146 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:33.696149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:33.696150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:33.696353 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:33.696387 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:33.696401 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:33.696664 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:33.696703 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:41:33.696729 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260809 00:41:33.697932 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:33.703037 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:33.699441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47151 (local address 127.0.0.1:36661)
0809 00:41:33.699473 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:33.699474 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:33.699578 (+ 104us) server_negotiation.cc:407] Connection header received
0809 00:41:33.699645 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:33.699645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:33.699655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:33.699715 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:33.699849 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:33.700573 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:33.701611 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:33.701917 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:33.702025 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:33.702760 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:33.702761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:33.702762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:33.702766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:33.702780 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:33.702785 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:33.702842 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:33.702943 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:41:33.702956 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:41:33.713923 30368 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:33.714473 30368 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:33.727602 30368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a404e61585604963acd0aa85285f9a63: Generated new TSK 0
I20260809 00:41:33.727936 30368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:33.732817 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30279
W20260809 00:41:33.740435 30264 connection.cc:535] client connection to 127.0.0.1:36661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36661): Connection reset by peer (error 104)
2026-08-09T00:41:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:33 dist-test-slave-1zqn krb5kdc[30385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:33 dist-test-slave-1zqn krb5kdc[30385](info): set up 2 sockets
Aug 09 00:41:33 dist-test-slave-1zqn krb5kdc[30385](info): commencing operation
krb5kdc: starting...
W20260809 00:41:34.839247 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:34 dist-test-slave-1zqn krb5kdc[30385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:34.967948 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:35.051460 30400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:35.051674 30400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:35.051697 30400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:35.052901 30400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:35.052943 30400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:35.052958 30400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:35.052970 30400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:35.052982 30400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:35.054435 30400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:35.054631 30400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:35.054805 30400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:35.059912 30407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:35.059901 30409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:35.059932 30406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:35.060117 30400 server_base.cc:1034] running on GCE node
I20260809 00:41:35.060523 30400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:35.060786 30400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:35.061926 30400 hybrid_clock.cc:648] HybridClock initialized: now 1786236095061907 us; error 37 us; skew 500 ppm
Aug 09 00:41:35 dist-test-slave-1zqn krb5kdc[30385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236095, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:35.064476 30400 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:35.064976 30400 webserver.cc:460] Webserver started at http://127.0.0.1:43145/ using document root <none> and password file <none>
I20260809 00:41:35.065268 30400 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:35.065351 30400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:35.065505 30400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:35.066566 30400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "edaad4490e4f4c2f90df8516d0f96ab5"
format_stamp: "Formatted at 2026-08-09 00:41:35 on dist-test-slave-1zqn"
server_key: "47fb702ec1f31e1e8bf3f5fb80814f87"
server_key_iv: "bd600a54548b0799c0963c216d25f58f"
server_key_version: "encryptionkey@0"
I20260809 00:41:35.066938 30400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "edaad4490e4f4c2f90df8516d0f96ab5"
format_stamp: "Formatted at 2026-08-09 00:41:35 on dist-test-slave-1zqn"
server_key: "47fb702ec1f31e1e8bf3f5fb80814f87"
server_key_iv: "bd600a54548b0799c0963c216d25f58f"
server_key_version: "encryptionkey@0"
I20260809 00:41:35.068419 30400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:41:35.069288 30416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:35.069474 30400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:35.069586 30400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "edaad4490e4f4c2f90df8516d0f96ab5"
format_stamp: "Formatted at 2026-08-09 00:41:35 on dist-test-slave-1zqn"
server_key: "47fb702ec1f31e1e8bf3f5fb80814f87"
server_key_iv: "bd600a54548b0799c0963c216d25f58f"
server_key_version: "encryptionkey@0"
I20260809 00:41:35.069681 30400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:35.073141 30400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:35.075239 30400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:35.075397 30400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:35.080262 30400 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40947
I20260809 00:41:35.080329 30476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40947 every 8 connection(s)
I20260809 00:41:35.080678 30400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:35.081745 30477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:35.082937 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30400
I20260809 00:41:35.083071 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:35.083253 7589 external_mini_cluster.cc:1428] Setting key 6dd15a04ebd93434a1d9dfd1aaab65ad
I20260809 00:41:35.084619 30477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5: Bootstrap starting.
I20260809 00:41:35.085467 30477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:35.085796 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:35.083785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52671 (local address 127.0.0.1:40947)
0809 00:41:35.083967 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:35.083977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:35.084008 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:41:35.084482 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:35.084489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:35.084514 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:35.084637 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:35.085323 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:35.085327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:35.085327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:35.085331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:35.085369 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:35.085379 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:35.085448 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:35.085599 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 00:41:35.085643 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260809 00:41:35.085817 30477 log.cc:824] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:35.086702 30477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5: No bootstrap required, opened a new log
I20260809 00:41:35.086724 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:35.088835 30477 raft_consensus.cc:348] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } }
I20260809 00:41:35.089018 30477 raft_consensus.cc:374] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:35.089099 30477 raft_consensus.cc:729] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edaad4490e4f4c2f90df8516d0f96ab5, State: Initialized, Role: FOLLOWER
I20260809 00:41:35.089249 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:35.088169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34541 (local address 127.0.0.1:40947)
0809 00:41:35.088211 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:35.088212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:35.088348 (+ 136us) server_negotiation.cc:407] Connection header received
0809 00:41:35.088408 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:35.088409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:35.088413 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:35.088460 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:35.088981 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:35.088982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:35.088982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:35.088986 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:35.088997 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:35.089000 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:35.089068 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:35.089155 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 00:41:35.089169 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:35.089236 30477 consensus_queue.cc:260] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } }
I20260809 00:41:35.089324 30477 raft_consensus.cc:388] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:35.089366 30477 raft_consensus.cc:482] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:35.089423 30477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:35.090070 30477 raft_consensus.cc:504] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } }
I20260809 00:41:35.090160 30477 leader_election.cc:302] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edaad4490e4f4c2f90df8516d0f96ab5; no voters:
I20260809 00:41:35.090286 30477 leader_election.cc:288] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:35.090341 30489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:35.090504 30489 raft_consensus.cc:686] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [term 1 LEADER]: Becoming Leader. State: Replica: edaad4490e4f4c2f90df8516d0f96ab5, State: Running, Role: LEADER
I20260809 00:41:35.090548 30477 sys_catalog.cc:564] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:35.090657 30489 consensus_queue.cc:237] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } }
I20260809 00:41:35.091130 30490 sys_catalog.cc:455] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } } }
I20260809 00:41:35.091135 30491 sys_catalog.cc:455] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edaad4490e4f4c2f90df8516d0f96ab5. Latest consensus state: current_term: 1 leader_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaad4490e4f4c2f90df8516d0f96ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40947 } } }
I20260809 00:41:35.091272 30491 sys_catalog.cc:458] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:35.091224 30490 sys_catalog.cc:458] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:35.092247 30494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:35.092747 30494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:35.094373 30494 catalog_manager.cc:1347] Generated new cluster ID: a374f660922a496d81ea1df31aff745e
I20260809 00:41:35.094452 30494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:35.104408 30494 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:35.104586 30494 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:35.116397 30494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edaad4490e4f4c2f90df8516d0f96ab5: Generated new TSK 0
I20260809 00:41:35.116619 30494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:35.118685 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30400
2026-08-09T00:41:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:35 dist-test-slave-1zqn krb5kdc[30510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:35 dist-test-slave-1zqn krb5kdc[30510](info): set up 2 sockets
Aug 09 00:41:35 dist-test-slave-1zqn krb5kdc[30510](info): commencing operation
krb5kdc: starting...
W20260809 00:41:36.205930 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:36.340142 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:36.424664 30524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:36.424948 30524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:36.424993 30524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:36.426224 30524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:36.426304 30524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:36.426329 30524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:36.426352 30524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:36.426383 30524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:36.427970 30524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:36.428337 30524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:36.428592 30524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:36.433715 30530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:36.433756 30532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:36.433759 30529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:36.434012 30524 server_base.cc:1034] running on GCE node
I20260809 00:41:36.434302 30524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:36.434604 30524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:36.435806 30524 hybrid_clock.cc:648] HybridClock initialized: now 1786236096435783 us; error 47 us; skew 500 ppm
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:36.438642 30524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:36.439191 30524 webserver.cc:460] Webserver started at http://127.0.0.1:43607/ using document root <none> and password file <none>
I20260809 00:41:36.439486 30524 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:36.439565 30524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:36.439730 30524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:36.440995 30524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "4e217dc76b47448dbaaf309b571adaff"
format_stamp: "Formatted at 2026-08-09 00:41:36 on dist-test-slave-1zqn"
server_key: "58fd45bc2b5d00ee8a09a70a8e6a3024"
server_key_iv: "d8e79281523b406dd1b0ce9edc6338d8"
server_key_version: "encryptionkey@0"
I20260809 00:41:36.441398 30524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "4e217dc76b47448dbaaf309b571adaff"
format_stamp: "Formatted at 2026-08-09 00:41:36 on dist-test-slave-1zqn"
server_key: "58fd45bc2b5d00ee8a09a70a8e6a3024"
server_key_iv: "d8e79281523b406dd1b0ce9edc6338d8"
server_key_version: "encryptionkey@0"
I20260809 00:41:36.442948 30524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:36.443840 30540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:36.444231 30524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:36.444342 30524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "4e217dc76b47448dbaaf309b571adaff"
format_stamp: "Formatted at 2026-08-09 00:41:36 on dist-test-slave-1zqn"
server_key: "58fd45bc2b5d00ee8a09a70a8e6a3024"
server_key_iv: "d8e79281523b406dd1b0ce9edc6338d8"
server_key_version: "encryptionkey@0"
I20260809 00:41:36.444439 30524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:36.488361 30524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:36.490577 30524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:36.490767 30524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:36.495012 30524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35545
I20260809 00:41:36.495070 30598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35545 every 8 connection(s)
I20260809 00:41:36.495461 30524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:36.496618 30600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:36.497152 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30524
I20260809 00:41:36.497279 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:36.497459 7589 external_mini_cluster.cc:1428] Setting key 72d76f9601772ac4a0238d20a4401a0e
I20260809 00:41:36.499519 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: Bootstrap starting.
I20260809 00:41:36.500236 30600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:36.500538 30600 log.cc:824] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:36.501379 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: No bootstrap required, opened a new log
I20260809 00:41:36.503921 30600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260809 00:41:36.504136 30600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:36.504185 30600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e217dc76b47448dbaaf309b571adaff, State: Initialized, Role: FOLLOWER
I20260809 00:41:36.504398 30600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260809 00:41:36.504505 30600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:36.504552 30600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:36.504602 30600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 0 FOLLOWER]: Advancing to term 1
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:41:36.505239 30600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260809 00:41:36.505363 30600 leader_election.cc:302] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e217dc76b47448dbaaf309b571adaff; no voters:
I20260809 00:41:36.505559 30600 leader_election.cc:288] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:36.505837 30600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:36.505604 30606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:36.506157 30606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [term 1 LEADER]: Becoming Leader. State: Replica: 4e217dc76b47448dbaaf309b571adaff, State: Running, Role: LEADER
I20260809 00:41:36.506264 30606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260809 00:41:36.507194 30607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e217dc76b47448dbaaf309b571adaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260809 00:41:36.507290 30607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:36.507712 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e217dc76b47448dbaaf309b571adaff. Latest consensus state: current_term: 1 leader_uuid: "4e217dc76b47448dbaaf309b571adaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e217dc76b47448dbaaf309b571adaff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260809 00:41:36.507834 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff [sys.catalog]: This master's current role is: LEADER
W20260809 00:41:36.508100 30621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:36.508158 30621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:36.508214 30622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:36.508425 30622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:36.509711 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:36.498033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40605 (local address 127.0.0.1:35545)
0809 00:41:36.498244 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:36.498252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:36.498277 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:41:36.498712 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:36.498719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:36.498766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:36.498913 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:36.499634 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:36.500766 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:36.501864 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:36.502315 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:36.502402 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:36.505321 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:36.505325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:36.505326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:36.505362 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:36.507552 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:36.507984 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:36.507988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:36.507989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:36.508131 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:36.508371 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:36.508372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:36.508373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:36.508701 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:36.508838 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:36.509319 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 00:41:36.509490 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260809 00:41:36.510079 30622 catalog_manager.cc:1347] Generated new cluster ID: bd45a35e51744a38a716c2956d6b7a79
I20260809 00:41:36.510126 30622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:36.513132 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:36.522181 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:36.514691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50235 (local address 127.0.0.1:35545)
0809 00:41:36.514725 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:36.514726 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:36.514842 (+ 116us) server_negotiation.cc:407] Connection header received
0809 00:41:36.514905 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:36.514906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:36.514916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:36.514980 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:36.515654 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:36.516430 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:36.517445 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:36.517749 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:36.517806 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:36.519296 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:36.519298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:36.519298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:36.519304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:36.520928 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:36.521351 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:36.521352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:36.521353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:36.521419 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:36.521629 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:36.521630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:36.521630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:36.521752 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:36.521841 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:36.522002 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:41:36.522091 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:41:36.522876 30622 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:36.523655 30622 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:36.529150 30622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e217dc76b47448dbaaf309b571adaff: Generated new TSK 0
I20260809 00:41:36.529366 30622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:36.534166 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30524
2026-08-09T00:41:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30634](info): set up 2 sockets
Aug 09 00:41:36 dist-test-slave-1zqn krb5kdc[30634](info): commencing operation
krb5kdc: starting...
W20260809 00:41:37.651252 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:37 dist-test-slave-1zqn krb5kdc[30634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:37.779685 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:37.863736 30648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:37.863996 30648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:37.864034 30648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:37.865293 30648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:37.865358 30648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:37.865382 30648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:37.865412 30648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:37.865444 30648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:37.866946 30648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:37.867235 30648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:37.867480 30648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:37.872637 30655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:37.872681 30654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:37.872772 30648 server_base.cc:1034] running on GCE node
W20260809 00:41:37.872845 30657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:37.873099 30648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:37.873459 30648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:37.874648 30648 hybrid_clock.cc:648] HybridClock initialized: now 1786236097874638 us; error 48 us; skew 500 ppm
Aug 09 00:41:37 dist-test-slave-1zqn krb5kdc[30634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:37.881009 30648 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:37.881487 30648 webserver.cc:460] Webserver started at http://127.0.0.1:46281/ using document root <none> and password file <none>
I20260809 00:41:37.881742 30648 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:37.881814 30648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:37.881968 30648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:37.883078 30648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "74833ea01cf348c4a7b791b50252bbad"
format_stamp: "Formatted at 2026-08-09 00:41:37 on dist-test-slave-1zqn"
server_key: "15e81b2326f533fababa05cb9c61fe2d"
server_key_iv: "793ddc956dd53c9439d1c624d5613895"
server_key_version: "encryptionkey@0"
I20260809 00:41:37.883486 30648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "74833ea01cf348c4a7b791b50252bbad"
format_stamp: "Formatted at 2026-08-09 00:41:37 on dist-test-slave-1zqn"
server_key: "15e81b2326f533fababa05cb9c61fe2d"
server_key_iv: "793ddc956dd53c9439d1c624d5613895"
server_key_version: "encryptionkey@0"
I20260809 00:41:37.885006 30648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:37.885794 30665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:37.885974 30648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:37.886082 30648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "74833ea01cf348c4a7b791b50252bbad"
format_stamp: "Formatted at 2026-08-09 00:41:37 on dist-test-slave-1zqn"
server_key: "15e81b2326f533fababa05cb9c61fe2d"
server_key_iv: "793ddc956dd53c9439d1c624d5613895"
server_key_version: "encryptionkey@0"
I20260809 00:41:37.886174 30648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:37.900549 30648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:37.902379 30648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:37.902549 30648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:37.907063 30648 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42313
I20260809 00:41:37.907099 30724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42313 every 8 connection(s)
I20260809 00:41:37.907469 30648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:37.908546 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:37.910786 30725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad: Bootstrap starting.
I20260809 00:41:37.911334 30725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:37.911569 30725 log.cc:824] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:37.912333 30725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad: No bootstrap required, opened a new log
I20260809 00:41:37.913888 30725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 00:41:37.914023 30725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:37.914050 30725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74833ea01cf348c4a7b791b50252bbad, State: Initialized, Role: FOLLOWER
I20260809 00:41:37.914186 30725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 00:41:37.914242 30725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:37.914263 30725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:37.914289 30725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:37.914763 30725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 00:41:37.914847 30725 leader_election.cc:302] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74833ea01cf348c4a7b791b50252bbad; no voters:
I20260809 00:41:37.914965 30725 leader_election.cc:288] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:37.915031 30728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:37.915207 30728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [term 1 LEADER]: Becoming Leader. State: Replica: 74833ea01cf348c4a7b791b50252bbad, State: Running, Role: LEADER
I20260809 00:41:37.915263 30725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:37.915318 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30648
I20260809 00:41:37.915357 30728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 00:41:37.915463 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:37.915684 7589 external_mini_cluster.cc:1428] Setting key 3fc231090cdf19d090902fe1b64bd407
I20260809 00:41:37.915717 30728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74833ea01cf348c4a7b791b50252bbad. Latest consensus state: current_term: 1 leader_uuid: "74833ea01cf348c4a7b791b50252bbad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } } }
I20260809 00:41:37.915728 30730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74833ea01cf348c4a7b791b50252bbad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74833ea01cf348c4a7b791b50252bbad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } } }
I20260809 00:41:37.915813 30728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:37.915817 30730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:37.916621 30733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:37.917214 30733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:37.918851 30733 catalog_manager.cc:1347] Generated new cluster ID: 5e0e45e3fafa48779f076090ae1b1b89
I20260809 00:41:37.918906 30733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:37.921631 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:37.916243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37629 (local address 127.0.0.1:42313)
0809 00:41:37.916504 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:37.916515 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:37.916805 (+ 290us) server_negotiation.cc:407] Connection header received
0809 00:41:37.916936 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:37.916943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:37.917002 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:37.917153 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:37.917531 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:37.918789 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:37.919648 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:37.920127 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:37.920255 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:37.920957 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:37.920961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:37.920961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:37.921147 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:37.921177 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:37.921188 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:37.921265 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:37.921339 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:41:37.921404 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
I20260809 00:41:37.922544 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:37.926949 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:37.924018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33183 (local address 127.0.0.1:42313)
0809 00:41:37.924078 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:37.924080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:37.924198 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:41:37.924261 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:37.924262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:37.924272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:37.924328 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:37.924462 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:37.924976 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:37.925710 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:37.925919 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:37.925998 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:37.926695 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:37.926696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:37.926697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:37.926701 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:37.926714 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:37.926719 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:37.926765 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:37.926822 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 00:41:37.926847 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:37.948794 30733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:37.949376 30733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:37.955771 30733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74833ea01cf348c4a7b791b50252bbad: Generated new TSK 0
I20260809 00:41:37.956004 30733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:37.961833 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30648
2026-08-09T00:41:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:38 dist-test-slave-1zqn krb5kdc[30761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:38 dist-test-slave-1zqn krb5kdc[30761](info): set up 2 sockets
Aug 09 00:41:38 dist-test-slave-1zqn krb5kdc[30761](info): commencing operation
krb5kdc: starting...
W20260809 00:41:39.055125 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:39 dist-test-slave-1zqn krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:39.183584 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:39.268386 30775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:39.268683 30775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:39.268725 30775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:39.269878 30775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:39.269943 30775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:39.269971 30775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:39.270002 30775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:39.270031 30775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:39.271544 30775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:39.271827 30775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:39.272125 30775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:39.277361 30782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:39.277376 30781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:39.277415 30784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:39.277420 30775 server_base.cc:1034] running on GCE node
I20260809 00:41:39.277849 30775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:39.278149 30775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:39.279316 30775 hybrid_clock.cc:648] HybridClock initialized: now 1786236099279299 us; error 49 us; skew 500 ppm
Aug 09 00:41:39 dist-test-slave-1zqn krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:39.282032 30775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:39.282562 30775 webserver.cc:460] Webserver started at http://127.0.0.1:36905/ using document root <none> and password file <none>
I20260809 00:41:39.282848 30775 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:39.282945 30775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:39.283105 30775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:39.284420 30775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "ef1d000dec2c4b4dad7d4c348d4421a9"
format_stamp: "Formatted at 2026-08-09 00:41:39 on dist-test-slave-1zqn"
server_key: "7dc7d19e7f1043b3937020370b2e814a"
server_key_iv: "ab27c841dedf71d5996ea72fe63ad0f6"
server_key_version: "encryptionkey@0"
I20260809 00:41:39.284821 30775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "ef1d000dec2c4b4dad7d4c348d4421a9"
format_stamp: "Formatted at 2026-08-09 00:41:39 on dist-test-slave-1zqn"
server_key: "7dc7d19e7f1043b3937020370b2e814a"
server_key_iv: "ab27c841dedf71d5996ea72fe63ad0f6"
server_key_version: "encryptionkey@0"
I20260809 00:41:39.286307 30775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:41:39.287113 30791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:39.287274 30775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:39.287402 30775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "ef1d000dec2c4b4dad7d4c348d4421a9"
format_stamp: "Formatted at 2026-08-09 00:41:39 on dist-test-slave-1zqn"
server_key: "7dc7d19e7f1043b3937020370b2e814a"
server_key_iv: "ab27c841dedf71d5996ea72fe63ad0f6"
server_key_version: "encryptionkey@0"
I20260809 00:41:39.287484 30775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:39.291289 30775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:39.293318 30775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:39.293511 30775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:39.298019 30775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37851
I20260809 00:41:39.298049 30846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37851 every 8 connection(s)
I20260809 00:41:39.298429 30775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:39.299052 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30775
I20260809 00:41:39.299157 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:39.299391 7589 external_mini_cluster.cc:1428] Setting key 57edfbb4553a6999b95a0a1d2104ab60
I20260809 00:41:39.299681 30847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:39.302168 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:39.300146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32781 (local address 127.0.0.1:37851)
0809 00:41:39.300345 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:39.300355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:39.300390 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:41:39.300841 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:39.300848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:39.300874 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:39.301000 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:39.301662 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:39.301667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:39.301667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:39.301672 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:39.301709 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:39.301720 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:39.301781 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:39.301873 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:41:39.301909 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260809 00:41:39.303335 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:39.303403 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9: Bootstrap starting.
I20260809 00:41:39.304175 30847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:39.304538 30847 log.cc:824] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:39.305306 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9: No bootstrap required, opened a new log
I20260809 00:41:39.306077 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:39.305002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56715 (local address 127.0.0.1:37851)
0809 00:41:39.305063 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:39.305066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:39.305211 (+ 145us) server_negotiation.cc:407] Connection header received
0809 00:41:39.305282 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:39.305283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:39.305287 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:39.305340 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:39.305889 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:39.305890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:39.305890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:39.305895 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:39.305907 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:39.305910 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:39.305956 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:39.305999 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 00:41:39.306012 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:41:39.307421 30847 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } }
I20260809 00:41:39.307552 30847 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:39.307575 30847 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1d000dec2c4b4dad7d4c348d4421a9, State: Initialized, Role: FOLLOWER
I20260809 00:41:39.307729 30847 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } }
I20260809 00:41:39.307816 30847 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:39.307861 30847 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:39.307914 30847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:39.308521 30847 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } }
I20260809 00:41:39.308642 30847 leader_election.cc:302] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef1d000dec2c4b4dad7d4c348d4421a9; no voters:
I20260809 00:41:39.308842 30847 leader_election.cc:288] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:39.308878 30859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:39.309048 30859 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [term 1 LEADER]: Becoming Leader. State: Replica: ef1d000dec2c4b4dad7d4c348d4421a9, State: Running, Role: LEADER
I20260809 00:41:39.309091 30847 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:39.309238 30859 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } }
I20260809 00:41:39.309643 30861 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef1d000dec2c4b4dad7d4c348d4421a9. Latest consensus state: current_term: 1 leader_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } } }
I20260809 00:41:39.309667 30860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1d000dec2c4b4dad7d4c348d4421a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37851 } } }
I20260809 00:41:39.309737 30861 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:39.309755 30860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:39.310196 30868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:39.310710 30868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:39.312218 30868 catalog_manager.cc:1347] Generated new cluster ID: 7708e691861c456fa64082e9ca0f7d4f
I20260809 00:41:39.312260 30868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:39.321681 30868 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:39.321892 30868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:39.331521 30868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef1d000dec2c4b4dad7d4c348d4421a9: Generated new TSK 0
I20260809 00:41:39.331753 30868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:39.333985 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30775
2026-08-09T00:41:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:39 dist-test-slave-1zqn krb5kdc[30881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:39 dist-test-slave-1zqn krb5kdc[30881](info): set up 2 sockets
Aug 09 00:41:39 dist-test-slave-1zqn krb5kdc[30881](info): commencing operation
krb5kdc: starting...
W20260809 00:41:40.472337 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:40.599625 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:40.683699 30895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:40.684062 30895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:40.684178 30895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:40.685377 30895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:40.685467 30895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:40.685496 30895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:40.685527 30895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:40.685550 30895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:40.687151 30895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:40.687448 30895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:40.687685 30895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:41:40.692960 30895 server_base.cc:1034] running on GCE node
W20260809 00:41:40.693076 30901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:40.692906 30900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:40.693117 30903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:40.693462 30895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:40.693770 30895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:40.694937 30895 hybrid_clock.cc:648] HybridClock initialized: now 1786236100694939 us; error 46 us; skew 500 ppm
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:40.697422 30895 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:40.697878 30895 webserver.cc:460] Webserver started at http://127.0.0.1:41807/ using document root <none> and password file <none>
I20260809 00:41:40.698125 30895 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:40.698199 30895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:40.698352 30895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:40.699429 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "0ac219cef8a847d1ba5b77d4dbc9779c"
format_stamp: "Formatted at 2026-08-09 00:41:40 on dist-test-slave-1zqn"
server_key: "4ea745eec4c032cebd30cf63866281f1"
server_key_iv: "58a6b2bca50afbdfdc3e38d2d7bf9d37"
server_key_version: "encryptionkey@0"
I20260809 00:41:40.699833 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "0ac219cef8a847d1ba5b77d4dbc9779c"
format_stamp: "Formatted at 2026-08-09 00:41:40 on dist-test-slave-1zqn"
server_key: "4ea745eec4c032cebd30cf63866281f1"
server_key_iv: "58a6b2bca50afbdfdc3e38d2d7bf9d37"
server_key_version: "encryptionkey@0"
I20260809 00:41:40.701283 30895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:40.702037 30911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:40.702219 30895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:40.702328 30895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "0ac219cef8a847d1ba5b77d4dbc9779c"
format_stamp: "Formatted at 2026-08-09 00:41:40 on dist-test-slave-1zqn"
server_key: "4ea745eec4c032cebd30cf63866281f1"
server_key_iv: "58a6b2bca50afbdfdc3e38d2d7bf9d37"
server_key_version: "encryptionkey@0"
I20260809 00:41:40.702402 30895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:40.718164 30895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:40.720355 30895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:40.720521 30895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:40.724514 30895 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45591
I20260809 00:41:40.724568 30965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45591 every 8 connection(s)
I20260809 00:41:40.724989 30895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:40.726068 30966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:40.728574 30966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: Bootstrap starting.
I20260809 00:41:40.729202 30966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:40.729501 30966 log.cc:824] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:40.730228 30966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: No bootstrap required, opened a new log
I20260809 00:41:40.731930 30966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } }
I20260809 00:41:40.732102 30966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:40.732152 30966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac219cef8a847d1ba5b77d4dbc9779c, State: Initialized, Role: FOLLOWER
I20260809 00:41:40.732350 30966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } }
I20260809 00:41:40.732436 30966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:40.732487 30966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:40.732542 30966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:40.733088 30966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } }
I20260809 00:41:40.733204 30966 leader_election.cc:302] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ac219cef8a847d1ba5b77d4dbc9779c; no voters:
I20260809 00:41:40.733372 30966 leader_election.cc:288] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:40.733498 30969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:40.733656 30966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:40.733722 30969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [term 1 LEADER]: Becoming Leader. State: Replica: 0ac219cef8a847d1ba5b77d4dbc9779c, State: Running, Role: LEADER
I20260809 00:41:40.733832 30969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } }
I20260809 00:41:40.734592 30971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ac219cef8a847d1ba5b77d4dbc9779c. Latest consensus state: current_term: 1 leader_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } } }
I20260809 00:41:40.734773 30971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:40.734782 30970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ac219cef8a847d1ba5b77d4dbc9779c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45591 } } }
I20260809 00:41:40.734908 30970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:40.735321 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 30895
I20260809 00:41:40.735458 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:40.735684 7589 external_mini_cluster.cc:1428] Setting key 648d6fc4eeea18e4971ae549ac48abdb
W20260809 00:41:40.735724 30988 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:40.735769 30988 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:40.735877 30989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:40.736136 30989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:40.737818 30989 catalog_manager.cc:1347] Generated new cluster ID: 4eb7c13a7fd44a7fa807b8dab4cb2b9e
I20260809 00:41:40.737871 30989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[30881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:41:40.746803 30989 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:40.747509 30989 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:40.748698 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:40.736425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:50415 (local address 127.0.0.1:45591)
0809 00:41:40.736651 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:40.736659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:40.736682 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:41:40.736775 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:40.736781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:40.736825 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:40.736926 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:40.737863 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:40.739453 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:40.740541 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:40.741299 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:40.741419 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:40.744571 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:40.744575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:40.744576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:40.744589 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:40.746747 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:40.747242 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:40.747245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:40.747246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:40.747345 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:40.747655 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:40.747656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:40.747657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:40.747993 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:40.748221 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:40.748323 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:41:40.748441 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260809 00:41:40.749774 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:40.759140 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:40.751234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:45243 (local address 127.0.0.1:45591)
0809 00:41:40.751271 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:40.751273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:40.751418 (+ 145us) server_negotiation.cc:407] Connection header received
0809 00:41:40.751450 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:40.751451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:40.751464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:40.751534 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:40.752131 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:40.753058 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:40.754083 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:40.754458 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:40.754537 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:40.755954 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:40.755956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:40.755956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:40.755964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:40.757700 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:40.758130 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:40.758132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:40.758132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:40.758223 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:40.758500 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:40.758501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:40.758501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:40.758620 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:40.758707 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:40.758952 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 00:41:40.759029 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:41:40.760946 30989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ac219cef8a847d1ba5b77d4dbc9779c: Generated new TSK 0
I20260809 00:41:40.761157 30989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:40.767447 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 30895
W20260809 00:41:40.775923 30879 connection.cc:535] client connection to 127.0.0.1:45591 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45591): Connection reset by peer (error 104)
2026-08-09T00:41:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[31001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[31001](info): set up 2 sockets
Aug 09 00:41:40 dist-test-slave-1zqn krb5kdc[31001](info): commencing operation
krb5kdc: starting...
W20260809 00:41:41.959401 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:42 dist-test-slave-1zqn krb5kdc[31001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:42.087311 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:42.171365 31014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:42.171572 31014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:42.171595 31014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:42.172766 31014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:42.172811 31014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:42.172825 31014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:42.172838 31014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:42.172894 31014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:42.174387 31014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:42.174647 31014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:42.174880 31014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:42.179955 31019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:42.180140 31014 server_base.cc:1034] running on GCE node
W20260809 00:41:42.179953 31022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:42.180246 31020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:42.180490 31014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:42.180814 31014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:42.181981 31014 hybrid_clock.cc:648] HybridClock initialized: now 1786236102181963 us; error 37 us; skew 500 ppm
Aug 09 00:41:42 dist-test-slave-1zqn krb5kdc[31001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:42.184474 31014 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:42.184964 31014 webserver.cc:460] Webserver started at http://127.0.0.1:46143/ using document root <none> and password file <none>
I20260809 00:41:42.185212 31014 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:42.185282 31014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:42.185436 31014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:42.186532 31014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "dc0cb7bc1fa84739954912779a5c9ab5"
format_stamp: "Formatted at 2026-08-09 00:41:42 on dist-test-slave-1zqn"
server_key: "20bb57e2f58c33bc1768e93075458645"
server_key_iv: "76a90a4f81030215fcc6e5b59ae62e1c"
server_key_version: "encryptionkey@0"
I20260809 00:41:42.186910 31014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "dc0cb7bc1fa84739954912779a5c9ab5"
format_stamp: "Formatted at 2026-08-09 00:41:42 on dist-test-slave-1zqn"
server_key: "20bb57e2f58c33bc1768e93075458645"
server_key_iv: "76a90a4f81030215fcc6e5b59ae62e1c"
server_key_version: "encryptionkey@0"
I20260809 00:41:42.188313 31014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:41:42.189045 31030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:42.189250 31014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:42.189360 31014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "dc0cb7bc1fa84739954912779a5c9ab5"
format_stamp: "Formatted at 2026-08-09 00:41:42 on dist-test-slave-1zqn"
server_key: "20bb57e2f58c33bc1768e93075458645"
server_key_iv: "76a90a4f81030215fcc6e5b59ae62e1c"
server_key_version: "encryptionkey@0"
I20260809 00:41:42.189433 31014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:42.199506 31014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:42.201416 31014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:42.201581 31014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:42.205814 31014 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33189
I20260809 00:41:42.205849 31091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33189 every 8 connection(s)
I20260809 00:41:42.206241 31014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:42.207269 31093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:42.209599 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5: Bootstrap starting.
I20260809 00:41:42.210166 31093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:42.210405 31093 log.cc:824] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:42.211071 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5: No bootstrap required, opened a new log
I20260809 00:41:42.212692 31093 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260809 00:41:42.212831 31093 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:42.212852 31093 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0cb7bc1fa84739954912779a5c9ab5, State: Initialized, Role: FOLLOWER
I20260809 00:41:42.213140 31093 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260809 00:41:42.213209 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31014
I20260809 00:41:42.213241 31093 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:42.213280 31093 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:42.213315 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:42.213337 31093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:42.213514 7589 external_mini_cluster.cc:1428] Setting key 0a917dc8dfa619963d42c31a5f6fac6f
I20260809 00:41:42.214033 31093 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260809 00:41:42.214162 31093 leader_election.cc:302] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc0cb7bc1fa84739954912779a5c9ab5; no voters:
I20260809 00:41:42.214336 31093 leader_election.cc:288] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:42.214361 31099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:42.214471 31099 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [term 1 LEADER]: Becoming Leader. State: Replica: dc0cb7bc1fa84739954912779a5c9ab5, State: Running, Role: LEADER
I20260809 00:41:42.214607 31093 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:42.214634 31099 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260809 00:41:42.215123 31100 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0cb7bc1fa84739954912779a5c9ab5. Latest consensus state: current_term: 1 leader_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260809 00:41:42.215211 31100 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:42.215420 31101 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cb7bc1fa84739954912779a5c9ab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260809 00:41:42.215729 31101 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:42.215873 31103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:42.216449 31103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:42.218143 31103 catalog_manager.cc:1347] Generated new cluster ID: 094ccb8a4ec24a45ae241d2b1360156a
I20260809 00:41:42.218201 31103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:42.219826 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:42.214011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:57371 (local address 127.0.0.1:33189)
0809 00:41:42.214262 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:42.214272 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:42.214311 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:41:42.214432 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:42.214440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:42.214505 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:42.214894 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:42.214993 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:42.216486 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:42.217589 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:42.218176 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:42.218325 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:42.219085 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:42.219089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:42.219089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:42.219323 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:42.219360 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:42.219373 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:42.219450 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:42.219584 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 00:41:42.219614 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":135,"threads_started":1}
I20260809 00:41:42.220813 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:42.225867 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:42.222369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:52119 (local address 127.0.0.1:33189)
0809 00:41:42.222401 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:42.222403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:42.222517 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:41:42.222577 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:42.222578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:42.222587 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:42.222637 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:42.222770 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:42.223488 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:42.224530 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:42.224830 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:42.224893 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:42.225595 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:42.225596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:42.225597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:42.225601 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:42.225615 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:42.225620 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:42.225676 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:42.225777 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:41:42.225789 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:41:42.227313 31103 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:42.227928 31103 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:42.230715 31103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc0cb7bc1fa84739954912779a5c9ab5: Generated new TSK 0
I20260809 00:41:42.230911 31103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:42.240109 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31014
W20260809 00:41:42.250721 30999 connection.cc:535] client connection to 127.0.0.1:33189 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33189): Connection reset by peer (error 104)
2026-08-09T00:41:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:42 dist-test-slave-1zqn krb5kdc[31129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:42 dist-test-slave-1zqn krb5kdc[31129](info): set up 2 sockets
Aug 09 00:41:42 dist-test-slave-1zqn krb5kdc[31129](info): commencing operation
krb5kdc: starting...
W20260809 00:41:43.343097 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:43 dist-test-slave-1zqn krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:43.473214 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:43.556419 31145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:43.556623 31145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:43.556646 31145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:43.557717 31145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:43.557752 31145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:43.557767 31145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:43.557780 31145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:43.557792 31145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:43.559324 31145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:43.559530 31145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:43.559696 31145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:43.564790 31154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:43.564817 31151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:43.564800 31152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:43.564834 31145 server_base.cc:1034] running on GCE node
I20260809 00:41:43.565328 31145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:43.565572 31145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:43.566712 31145 hybrid_clock.cc:648] HybridClock initialized: now 1786236103566703 us; error 33 us; skew 500 ppm
Aug 09 00:41:43 dist-test-slave-1zqn krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236103, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:43.572760 31145 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:43.573259 31145 webserver.cc:460] Webserver started at http://127.0.0.1:39099/ using document root <none> and password file <none>
I20260809 00:41:43.573515 31145 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:43.573562 31145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:43.573737 31145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:43.574819 31145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f19bb5737ce648a185854c3e5b3d6140"
format_stamp: "Formatted at 2026-08-09 00:41:43 on dist-test-slave-1zqn"
server_key: "e45e0286d7c36fe7287f01f88f2419a4"
server_key_iv: "857d80aa20971c13e0fdf8cb2ce0f17e"
server_key_version: "encryptionkey@0"
I20260809 00:41:43.575187 31145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f19bb5737ce648a185854c3e5b3d6140"
format_stamp: "Formatted at 2026-08-09 00:41:43 on dist-test-slave-1zqn"
server_key: "e45e0286d7c36fe7287f01f88f2419a4"
server_key_iv: "857d80aa20971c13e0fdf8cb2ce0f17e"
server_key_version: "encryptionkey@0"
I20260809 00:41:43.576624 31145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:43.577385 31162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:43.577613 31145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:43.577715 31145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f19bb5737ce648a185854c3e5b3d6140"
format_stamp: "Formatted at 2026-08-09 00:41:43 on dist-test-slave-1zqn"
server_key: "e45e0286d7c36fe7287f01f88f2419a4"
server_key_iv: "857d80aa20971c13e0fdf8cb2ce0f17e"
server_key_version: "encryptionkey@0"
I20260809 00:41:43.577795 31145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:43.581403 31145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:43.583323 31145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:43.583479 31145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:43.587740 31145 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44691
I20260809 00:41:43.587795 31216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44691 every 8 connection(s)
I20260809 00:41:43.588192 31145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:43.588491 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31145
I20260809 00:41:43.588651 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:43.588855 7589 external_mini_cluster.cc:1428] Setting key ce7428acfde945cd02552bd2a50e338e
I20260809 00:41:43.589385 31217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:43.591514 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:43.589451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:38153 (local address 127.0.0.1:44691)
0809 00:41:43.589677 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:43.589686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:43.589711 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:41:43.590163 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:43.590170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:43.590195 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:43.590351 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:43.591059 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:43.591063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:43.591063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:43.591068 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:43.591141 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:43.591153 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:43.591229 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:43.591299 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 00:41:43.591346 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260809 00:41:43.592562 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:43.593001 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140: Bootstrap starting.
I20260809 00:41:43.593691 31217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:43.594056 31217 log.cc:824] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:43.594971 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140: No bootstrap required, opened a new log
I20260809 00:41:43.595115 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:43.594006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:54689 (local address 127.0.0.1:44691)
0809 00:41:43.594048 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:43.594050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:43.594095 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:41:43.594181 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:43.594182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:43.594187 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:43.594244 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:43.594822 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:43.594823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:43.594824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:43.594827 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:43.594840 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:43.594843 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:43.594893 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:43.594967 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:41:43.594977 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:43.596948 31217 raft_consensus.cc:348] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260809 00:41:43.597118 31217 raft_consensus.cc:374] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:43.597169 31217 raft_consensus.cc:729] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f19bb5737ce648a185854c3e5b3d6140, State: Initialized, Role: FOLLOWER
I20260809 00:41:43.597297 31217 consensus_queue.cc:260] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260809 00:41:43.597396 31217 raft_consensus.cc:388] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:43.597440 31217 raft_consensus.cc:482] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:43.597493 31217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:43.598053 31217 raft_consensus.cc:504] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260809 00:41:43.598184 31217 leader_election.cc:302] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f19bb5737ce648a185854c3e5b3d6140; no voters:
I20260809 00:41:43.598388 31217 leader_election.cc:288] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:43.598426 31227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:43.598601 31227 raft_consensus.cc:686] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [term 1 LEADER]: Becoming Leader. State: Replica: f19bb5737ce648a185854c3e5b3d6140, State: Running, Role: LEADER
I20260809 00:41:43.598693 31217 sys_catalog.cc:564] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:43.598781 31227 consensus_queue.cc:237] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260809 00:41:43.599185 31228 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f19bb5737ce648a185854c3e5b3d6140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } } }
I20260809 00:41:43.599195 31229 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f19bb5737ce648a185854c3e5b3d6140. Latest consensus state: current_term: 1 leader_uuid: "f19bb5737ce648a185854c3e5b3d6140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19bb5737ce648a185854c3e5b3d6140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } } }
I20260809 00:41:43.599310 31228 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:43.599584 31229 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:43.599627 31236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:43.600250 31236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:43.601775 31236 catalog_manager.cc:1347] Generated new cluster ID: a46751454b4c45caa03aba877e1df6b3
I20260809 00:41:43.601828 31236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:43.605193 31236 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:43.605350 31236 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:43.611249 31236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f19bb5737ce648a185854c3e5b3d6140: Generated new TSK 0
I20260809 00:41:43.611438 31236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:43.614748 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31145
2026-08-09T00:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:43 dist-test-slave-1zqn krb5kdc[31251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:43 dist-test-slave-1zqn krb5kdc[31251](info): set up 2 sockets
Aug 09 00:41:43 dist-test-slave-1zqn krb5kdc[31251](info): commencing operation
krb5kdc: starting...
W20260809 00:41:44.714350 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:44 dist-test-slave-1zqn krb5kdc[31251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:44.843726 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:44.927192 31265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:44.927451 31265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:44.927487 31265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:44.928678 31265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:44.928755 31265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:44.928781 31265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:44.928802 31265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:44.928831 31265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:44.930318 31265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:44.930634 31265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:44.930876 31265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:44.936221 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:44.936224 31275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:44.936247 31273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:44.936532 31265 server_base.cc:1034] running on GCE node
I20260809 00:41:44.936784 31265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:44.937084 31265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:44.938261 31265 hybrid_clock.cc:648] HybridClock initialized: now 1786236104938257 us; error 39 us; skew 500 ppm
Aug 09 00:41:44 dist-test-slave-1zqn krb5kdc[31251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:44.940799 31265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:44.941272 31265 webserver.cc:460] Webserver started at http://127.0.0.1:39925/ using document root <none> and password file <none>
I20260809 00:41:44.941510 31265 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:44.941578 31265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:44.941735 31265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:44.942811 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe"
format_stamp: "Formatted at 2026-08-09 00:41:44 on dist-test-slave-1zqn"
server_key: "229caebfda4c5cd4c340427ef34c5191"
server_key_iv: "3d7c2d7d3924edf9cf0ce5ab45ad657c"
server_key_version: "encryptionkey@0"
I20260809 00:41:44.943192 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe"
format_stamp: "Formatted at 2026-08-09 00:41:44 on dist-test-slave-1zqn"
server_key: "229caebfda4c5cd4c340427ef34c5191"
server_key_iv: "3d7c2d7d3924edf9cf0ce5ab45ad657c"
server_key_version: "encryptionkey@0"
I20260809 00:41:44.944667 31265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:41:44.945408 31285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:44.945577 31265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:44.945690 31265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe"
format_stamp: "Formatted at 2026-08-09 00:41:44 on dist-test-slave-1zqn"
server_key: "229caebfda4c5cd4c340427ef34c5191"
server_key_iv: "3d7c2d7d3924edf9cf0ce5ab45ad657c"
server_key_version: "encryptionkey@0"
I20260809 00:41:44.945765 31265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:44.960506 31265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:44.962320 31265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:44.962486 31265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:44.966593 31265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38719
I20260809 00:41:44.966661 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38719 every 8 connection(s)
I20260809 00:41:44.967024 31265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:44.968127 31347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:44.969640 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31265
I20260809 00:41:44.969739 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:44.969929 7589 external_mini_cluster.cc:1428] Setting key 08b68495f06676fee96a6854d9667bbb
I20260809 00:41:44.971024 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe: Bootstrap starting.
I20260809 00:41:44.971778 31347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:44.972149 31347 log.cc:824] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:44.973024 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe: No bootstrap required, opened a new log
I20260809 00:41:44.975376 31347 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260809 00:41:44.975565 31347 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:44.975615 31347 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6eeb3e0b4b04e368ae3b6ed8a9a74fe, State: Initialized, Role: FOLLOWER
I20260809 00:41:44.975765 31347 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260809 00:41:44.975862 31347 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:44.975886 31347 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:44.975967 31347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:44.976606 31347 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260809 00:41:44.976699 31347 leader_election.cc:302] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6eeb3e0b4b04e368ae3b6ed8a9a74fe; no voters:
I20260809 00:41:44.976897 31347 leader_election.cc:288] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:44.976923 31354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:44.977061 31354 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [term 1 LEADER]: Becoming Leader. State: Replica: d6eeb3e0b4b04e368ae3b6ed8a9a74fe, State: Running, Role: LEADER
I20260809 00:41:44.977188 31347 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:44.977212 31354 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
Aug 09 00:41:44 dist-test-slave-1zqn krb5kdc[31251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:41:44.977685 31354 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6eeb3e0b4b04e368ae3b6ed8a9a74fe. Latest consensus state: current_term: 1 leader_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } } }
I20260809 00:41:44.977694 31355 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6eeb3e0b4b04e368ae3b6ed8a9a74fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } } }
I20260809 00:41:44.977927 31355 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:44.978143 31354 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:44.978341 31363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:44.979084 31363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:44.980940 31363 catalog_manager.cc:1347] Generated new cluster ID: 6d9047ec2b43492a88ad51af4248ace8
I20260809 00:41:44.980999 31363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:44.982192 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:44.970520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:58925 (local address 127.0.0.1:38719)
0809 00:41:44.970724 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:44.970734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:44.970769 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:41:44.971313 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:44.971324 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:44.971395 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:44.971558 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:44.972303 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:44.973689 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:44.974454 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:44.975161 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:44.975280 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:44.978197 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:44.978201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:44.978202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:44.978215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:44.980394 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:44.980813 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:44.980816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:44.980816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:44.980948 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:44.981262 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:44.981263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:44.981263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:44.981551 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:44.981705 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:44.981899 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 00:41:44.982000 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260809 00:41:44.983419 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:44.992455 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:44.984980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:32867 (local address 127.0.0.1:38719)
0809 00:41:44.985018 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:44.985020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:44.985149 (+ 129us) server_negotiation.cc:407] Connection header received
0809 00:41:44.985207 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:44.985208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:44.985218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:44.985270 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:44.985924 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:44.986663 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:44.987739 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:44.988092 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:44.988152 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:44.989602 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:44.989603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:44.989603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:44.989610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:44.991171 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:44.991617 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:44.991618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:44.991618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:44.991672 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:44.991910 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:44.991911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:44.991911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:44.992029 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:44.992159 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:44.992300 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 00:41:44.992363 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:45.002194 31363 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:45.002789 31363 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:45.008527 31363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6eeb3e0b4b04e368ae3b6ed8a9a74fe: Generated new TSK 0
I20260809 00:41:45.008782 31363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:45.015468 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31265
W20260809 00:41:45.023290 31248 connection.cc:535] client connection to 127.0.0.1:38719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38719): Connection reset by peer (error 104)
2026-08-09T00:41:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:45 dist-test-slave-1zqn krb5kdc[31381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:45 dist-test-slave-1zqn krb5kdc[31381](info): set up 2 sockets
Aug 09 00:41:45 dist-test-slave-1zqn krb5kdc[31381](info): commencing operation
krb5kdc: starting...
W20260809 00:41:46.107033 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:46 dist-test-slave-1zqn krb5kdc[31381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:46.237059 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:46.328711 31396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:46.328984 31396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:46.329028 31396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:46.330173 31396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:46.330247 31396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:46.330274 31396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:46.330298 31396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:46.330331 31396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:46.332389 31396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:46.332682 31396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:46.332985 31396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:46.338470 31402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:46.338471 31404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:46.338442 31401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:46.338760 31396 server_base.cc:1034] running on GCE node
I20260809 00:41:46.339095 31396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:46.339455 31396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:46.340636 31396 hybrid_clock.cc:648] HybridClock initialized: now 1786236106340605 us; error 42 us; skew 500 ppm
Aug 09 00:41:46 dist-test-slave-1zqn krb5kdc[31381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:46.343511 31396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:46.344012 31396 webserver.cc:460] Webserver started at http://127.0.0.1:35685/ using document root <none> and password file <none>
I20260809 00:41:46.344292 31396 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:46.344343 31396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:46.344444 31396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:46.345489 31396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "14981615746e4ec3b3b3bf2a99d8051e"
format_stamp: "Formatted at 2026-08-09 00:41:46 on dist-test-slave-1zqn"
server_key: "b4a1aa077efa3324ba81195f1c784564"
server_key_iv: "8a0f866429b739b975276c528a6ab811"
server_key_version: "encryptionkey@0"
I20260809 00:41:46.345808 31396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "14981615746e4ec3b3b3bf2a99d8051e"
format_stamp: "Formatted at 2026-08-09 00:41:46 on dist-test-slave-1zqn"
server_key: "b4a1aa077efa3324ba81195f1c784564"
server_key_iv: "8a0f866429b739b975276c528a6ab811"
server_key_version: "encryptionkey@0"
I20260809 00:41:46.347158 31396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:46.347934 31412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:46.348141 31396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:46.348243 31396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "14981615746e4ec3b3b3bf2a99d8051e"
format_stamp: "Formatted at 2026-08-09 00:41:46 on dist-test-slave-1zqn"
server_key: "b4a1aa077efa3324ba81195f1c784564"
server_key_iv: "8a0f866429b739b975276c528a6ab811"
server_key_version: "encryptionkey@0"
I20260809 00:41:46.348301 31396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:46.357965 31396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:46.360309 31396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:46.360426 31396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:46.364399 31396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46525
I20260809 00:41:46.364491 31470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46525 every 8 connection(s)
I20260809 00:41:46.364820 31396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:46.365748 31471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:46.368162 31471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: Bootstrap starting.
I20260809 00:41:46.368762 31471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:46.369053 31471 log.cc:824] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:46.369760 31471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: No bootstrap required, opened a new log
I20260809 00:41:46.371383 31471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } }
I20260809 00:41:46.371546 31471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:46.371598 31471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14981615746e4ec3b3b3bf2a99d8051e, State: Initialized, Role: FOLLOWER
I20260809 00:41:46.371747 31471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } }
I20260809 00:41:46.371830 31471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:46.371876 31471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:46.371928 31471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:46.372537 31471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } }
I20260809 00:41:46.372661 31471 leader_election.cc:302] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14981615746e4ec3b3b3bf2a99d8051e; no voters:
I20260809 00:41:46.372857 31471 leader_election.cc:288] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:46.372887 31474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:46.373044 31474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [term 1 LEADER]: Becoming Leader. State: Replica: 14981615746e4ec3b3b3bf2a99d8051e, State: Running, Role: LEADER
I20260809 00:41:46.373144 31471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:46.373135 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31396
I20260809 00:41:46.373255 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:46.373195 31474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } }
I20260809 00:41:46.373462 7589 external_mini_cluster.cc:1428] Setting key 9e8b802d54d0190e90ab337536526f4e
I20260809 00:41:46.374001 31475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14981615746e4ec3b3b3bf2a99d8051e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } } }
I20260809 00:41:46.374099 31475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:46.374423 31477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14981615746e4ec3b3b3bf2a99d8051e. Latest consensus state: current_term: 1 leader_uuid: "14981615746e4ec3b3b3bf2a99d8051e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14981615746e4ec3b3b3bf2a99d8051e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46525 } } }
I20260809 00:41:46.374580 31477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e [sys.catalog]: This master's current role is: LEADER
W20260809 00:41:46.375394 31495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:46.375458 31495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:46.375654 31486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:46.375851 31486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:46.377414 31486 catalog_manager.cc:1347] Generated new cluster ID: 275b0a3763b749409b86c9228a8106a3
I20260809 00:41:46.377470 31486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:46.380719 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:46.374202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:51965 (local address 127.0.0.1:46525)
0809 00:41:46.375134 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:46.375143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:46.375183 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:41:46.375301 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:46.375308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:46.375370 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:46.375554 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:46.375820 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:46.377266 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:46.378436 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:46.379066 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:46.379205 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:46.379953 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:46.379957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:46.379957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:46.380228 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:46.380270 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:46.380287 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:46.380381 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:46.380470 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 00:41:46.380504 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":64,"threads_started":1}
I20260809 00:41:46.382774 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:46.388067 31486 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:46.388370 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:46.384584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:37629 (local address 127.0.0.1:46525)
0809 00:41:46.384626 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:46.384628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:46.384638 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:46.384679 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:46.384680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:46.384690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:46.384759 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:46.384873 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:46.385617 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:46.386742 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:46.387062 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:46.387146 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:46.387872 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:46.387873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:46.387873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:46.387878 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:46.387895 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:46.387899 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:46.387965 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:46.388276 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 00:41:46.388291 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:41:46.388751 31486 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:46.399024 31486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14981615746e4ec3b3b3bf2a99d8051e: Generated new TSK 0
I20260809 00:41:46.399233 31486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:46.405802 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31396
W20260809 00:41:46.413475 31379 connection.cc:535] client connection to 127.0.0.1:46525 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46525): Connection reset by peer (error 104)
2026-08-09T00:41:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:46 dist-test-slave-1zqn krb5kdc[31506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:46 dist-test-slave-1zqn krb5kdc[31506](info): set up 2 sockets
Aug 09 00:41:46 dist-test-slave-1zqn krb5kdc[31506](info): commencing operation
krb5kdc: starting...
W20260809 00:41:47.571143 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:47 dist-test-slave-1zqn krb5kdc[31506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:41:47.703274 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:47.787667 31520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:47.787911 31520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:47.787963 31520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:47.789158 31520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:47.789240 31520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:47.789265 31520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:47.789299 31520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:47.789326 31520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:47.790899 31520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:47.791193 31520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:47.791447 31520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:47.796608 31526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:47.796591 31525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:47.796593 31528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:47.796882 31520 server_base.cc:1034] running on GCE node
I20260809 00:41:47.797238 31520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:47.797500 31520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:47.798655 31520 hybrid_clock.cc:648] HybridClock initialized: now 1786236107798631 us; error 35 us; skew 500 ppm
Aug 09 00:41:47 dist-test-slave-1zqn krb5kdc[31506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:47.801249 31520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:41:47.801738 31520 webserver.cc:460] Webserver started at http://127.0.0.1:41019/ using document root <none> and password file <none>
I20260809 00:41:47.802027 31520 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:47.802075 31520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:47.802230 31520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:47.803308 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f3c102b3e648498685bf1b71d1a5835c"
format_stamp: "Formatted at 2026-08-09 00:41:47 on dist-test-slave-1zqn"
server_key: "0611fe44d9884143237b4981b4989b1d"
server_key_iv: "c3a8e00d8d507f330e958f85340ff0b0"
server_key_version: "encryptionkey@0"
I20260809 00:41:47.803673 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f3c102b3e648498685bf1b71d1a5835c"
format_stamp: "Formatted at 2026-08-09 00:41:47 on dist-test-slave-1zqn"
server_key: "0611fe44d9884143237b4981b4989b1d"
server_key_iv: "c3a8e00d8d507f330e958f85340ff0b0"
server_key_version: "encryptionkey@0"
I20260809 00:41:47.805091 31520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:47.805809 31536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:47.805981 31520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:47.806094 31520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f3c102b3e648498685bf1b71d1a5835c"
format_stamp: "Formatted at 2026-08-09 00:41:47 on dist-test-slave-1zqn"
server_key: "0611fe44d9884143237b4981b4989b1d"
server_key_iv: "c3a8e00d8d507f330e958f85340ff0b0"
server_key_version: "encryptionkey@0"
I20260809 00:41:47.806172 31520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:47.809693 31520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:47.811539 31520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:47.811718 31520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:47.815964 31520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41443
I20260809 00:41:47.815995 31596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41443 every 8 connection(s)
I20260809 00:41:47.816365 31520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:47.817390 31597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:47.818885 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31520
I20260809 00:41:47.818984 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:47.819193 7589 external_mini_cluster.cc:1428] Setting key 2c3bd46ef3a26b69095163ab9eb2b137
I20260809 00:41:47.820278 31597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c: Bootstrap starting.
I20260809 00:41:47.821048 31597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:47.821411 31597 log.cc:824] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:47.821710 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:47.819701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:40099 (local address 127.0.0.1:41443)
0809 00:41:47.819888 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:47.819896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:47.819922 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:41:47.820424 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:47.820439 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:47.820467 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:47.820584 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:47.821267 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:47.821272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:47.821272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:47.821276 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:47.821318 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:47.821334 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:47.821419 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:47.821461 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:41:47.821506 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260809 00:41:47.822232 31597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c: No bootstrap required, opened a new log
I20260809 00:41:47.822664 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:47.824340 31597 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } }
I20260809 00:41:47.824501 31597 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:47.824563 31597 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c102b3e648498685bf1b71d1a5835c, State: Initialized, Role: FOLLOWER
I20260809 00:41:47.824707 31597 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } }
I20260809 00:41:47.824815 31597 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:47.824862 31597 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:47.824922 31597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:47.825250 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:47.824110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:55437 (local address 127.0.0.1:41443)
0809 00:41:47.824147 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:47.824149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:47.824282 (+ 133us) server_negotiation.cc:407] Connection header received
0809 00:41:47.824362 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:47.824363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:47.824367 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:47.824424 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:47.825043 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:47.825045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:47.825045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:47.825050 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:47.825064 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:47.825069 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:47.825137 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:47.825178 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 00:41:47.825191 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:41:47.825730 31597 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } }
I20260809 00:41:47.825845 31597 leader_election.cc:302] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3c102b3e648498685bf1b71d1a5835c; no voters:
I20260809 00:41:47.825969 31597 leader_election.cc:288] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:47.826030 31607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:47.826223 31607 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [term 1 LEADER]: Becoming Leader. State: Replica: f3c102b3e648498685bf1b71d1a5835c, State: Running, Role: LEADER
I20260809 00:41:47.826231 31597 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:47.826376 31607 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } }
I20260809 00:41:47.826808 31608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3c102b3e648498685bf1b71d1a5835c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } } }
I20260809 00:41:47.826921 31608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:47.827132 31616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:47.827452 31609 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3c102b3e648498685bf1b71d1a5835c. Latest consensus state: current_term: 1 leader_uuid: "f3c102b3e648498685bf1b71d1a5835c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c102b3e648498685bf1b71d1a5835c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41443 } } }
I20260809 00:41:47.827641 31616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:47.827662 31609 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:47.833083 31616 catalog_manager.cc:1347] Generated new cluster ID: b1918e17ac1a4ce6a368fb2e347297f6
I20260809 00:41:47.833158 31616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:47.845309 31616 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:47.845499 31616 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:47.850623 31616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3c102b3e648498685bf1b71d1a5835c: Generated new TSK 0
I20260809 00:41:47.850811 31616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:47.856001 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31520
2026-08-09T00:41:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:47 dist-test-slave-1zqn krb5kdc[31630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:47 dist-test-slave-1zqn krb5kdc[31630](info): set up 2 sockets
Aug 09 00:41:47 dist-test-slave-1zqn krb5kdc[31630](info): commencing operation
krb5kdc: starting...
W20260809 00:41:48.950968 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:49.082885 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36021
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:36021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:49.167323 31643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:49.167593 31643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:49.167637 31643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:49.168911 31643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:49.168990 31643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:49.169015 31643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:49.169044 31643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:49.169102 31643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:49.170624 31643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:36021
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:49.170926 31643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:49.171183 31643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:41:49.176352 31643 server_base.cc:1034] running on GCE node
W20260809 00:41:49.176334 31651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:49.176319 31648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:49.176334 31649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:49.176824 31643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:49.177203 31643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:49.178380 31643 hybrid_clock.cc:648] HybridClock initialized: now 1786236109178370 us; error 55 us; skew 500 ppm
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236109, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:49.185539 31643 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:49.186008 31643 webserver.cc:460] Webserver started at http://127.7.105.126:35737/ using document root <none> and password file <none>
I20260809 00:41:49.186247 31643 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:49.186292 31643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:49.186388 31643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:49.187430 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "355f3dae2329403e8851884d7736b203"
format_stamp: "Formatted at 2026-08-09 00:41:49 on dist-test-slave-1zqn"
server_key: "02fcdcb18cec493eb8bc3532c7fbe376"
server_key_iv: "11b36c3d0fa877565eea26ac51278e21"
server_key_version: "encryptionkey@0"
I20260809 00:41:49.187744 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "355f3dae2329403e8851884d7736b203"
format_stamp: "Formatted at 2026-08-09 00:41:49 on dist-test-slave-1zqn"
server_key: "02fcdcb18cec493eb8bc3532c7fbe376"
server_key_iv: "11b36c3d0fa877565eea26ac51278e21"
server_key_version: "encryptionkey@0"
I20260809 00:41:49.189182 31643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:49.189962 31658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:49.190200 31643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:41:49.190301 31643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "355f3dae2329403e8851884d7736b203"
format_stamp: "Formatted at 2026-08-09 00:41:49 on dist-test-slave-1zqn"
server_key: "02fcdcb18cec493eb8bc3532c7fbe376"
server_key_iv: "11b36c3d0fa877565eea26ac51278e21"
server_key_version: "encryptionkey@0"
I20260809 00:41:49.190382 31643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:49.230886 31643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:49.233039 31643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:49.233214 31643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:49.237519 31643 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:36021
I20260809 00:41:49.237579 31715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:36021 every 8 connection(s)
I20260809 00:41:49.237947 31643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:49.238884 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:49.239306 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31643
I20260809 00:41:49.239437 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:49.239634 7589 external_mini_cluster.cc:1428] Setting key 28d6f69ba6c6631492961f18edd1c95c
I20260809 00:41:49.242170 31716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203: Bootstrap starting.
I20260809 00:41:49.242856 31716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:49.243197 31716 log.cc:824] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:49.243994 31716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203: No bootstrap required, opened a new log
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:49.246771 31716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } }
I20260809 00:41:49.246934 31716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:49.246973 31716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 355f3dae2329403e8851884d7736b203, State: Initialized, Role: FOLLOWER
I20260809 00:41:49.247160 31716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } }
I20260809 00:41:49.247267 31716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:49.247314 31716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:49.247364 31716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:49.247956 31716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } }
I20260809 00:41:49.248098 31716 leader_election.cc:302] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 355f3dae2329403e8851884d7736b203; no voters:
I20260809 00:41:49.248312 31716 leader_election.cc:288] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:49.248373 31721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:49.248562 31721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [term 1 LEADER]: Becoming Leader. State: Replica: 355f3dae2329403e8851884d7736b203, State: Running, Role: LEADER
I20260809 00:41:49.248627 31716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:49.248692 31721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } }
I20260809 00:41:49.249066 31722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "355f3dae2329403e8851884d7736b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } } }
I20260809 00:41:49.249065 31721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 355f3dae2329403e8851884d7736b203. Latest consensus state: current_term: 1 leader_uuid: "355f3dae2329403e8851884d7736b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355f3dae2329403e8851884d7736b203" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 36021 } } }
I20260809 00:41:49.249236 31722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:49.249243 31721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:49.249477 31729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:49.250032 31729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:49.251672 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:49.240191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32859 (local address 127.7.105.126:36021)
0809 00:41:49.240369 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:49.240385 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:49.240412 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:41:49.240838 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:49.240844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:49.240889 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:49.241006 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:49.241849 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:49.242791 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:49.243799 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:49.244280 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:49.244373 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:49.247141 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:49.247145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:49.247145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:49.247156 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:49.249374 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:49.249915 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:49.249920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:49.249920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:49.250037 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:49.250430 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:49.250432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:49.250432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:49.250759 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:49.251276 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:49.251362 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:41:49.251463 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260809 00:41:49.252511 31729 catalog_manager.cc:1347] Generated new cluster ID: dae8b1b6e8c14dd6ac23b7e84b85c559
I20260809 00:41:49.252584 31729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:49.252738 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:49.261562 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:49.254213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34529 (local address 127.7.105.126:36021)
0809 00:41:49.254259 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:49.254261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:49.254332 (+ 71us) server_negotiation.cc:407] Connection header received
0809 00:41:49.254365 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:49.254366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:49.254377 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:49.254442 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:49.255023 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:49.255777 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:49.256791 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:49.257166 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:49.257241 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:49.258596 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:49.258598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:49.258598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:49.258606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:49.260289 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:49.260723 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:49.260724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:49.260725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:49.260788 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:49.261019 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:49.261020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:49.261020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:49.261167 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:49.261274 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:49.261393 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 00:41:49.261462 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:49.265301 31729 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:49.266001 31729 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:49.268415 31729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 355f3dae2329403e8851884d7736b203: Generated new TSK 0
I20260809 00:41:49.268620 31729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:49.272007 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31643
W20260809 00:41:49.278755 31628 connection.cc:535] client connection to 127.7.105.126:36021 recv error: Network error: failed to read from TLS socket (remote: 127.7.105.126:36021): Connection reset by peer (error 104)
2026-08-09T00:41:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31751](info): set up 2 sockets
Aug 09 00:41:49 dist-test-slave-1zqn krb5kdc[31751](info): commencing operation
krb5kdc: starting...
W20260809 00:41:50.370774 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:50 dist-test-slave-1zqn krb5kdc[31751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:50.503393 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44579
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:44579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:50.587118 31764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:50.587332 31764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:50.587356 31764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:50.588567 31764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:50.588620 31764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:50.588635 31764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:50.588649 31764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:50.588660 31764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:50.590188 31764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:44579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:44579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:50.590453 31764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:50.590737 31764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:50.595772 31770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:50.595939 31772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:50.595782 31774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:50.596289 31764 server_base.cc:1034] running on GCE node
I20260809 00:41:50.596534 31764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:50.596864 31764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:50.598021 31764 hybrid_clock.cc:648] HybridClock initialized: now 1786236110597990 us; error 48 us; skew 500 ppm
Aug 09 00:41:50 dist-test-slave-1zqn krb5kdc[31751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236110, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:50.600698 31764 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:50.601194 31764 webserver.cc:460] Webserver started at http://127.7.105.126:45853/ using document root <none> and password file <none>
I20260809 00:41:50.601449 31764 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:50.601495 31764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:50.601656 31764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:50.602738 31764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f8fc780d41494b0bba46d99fc59b40dd"
format_stamp: "Formatted at 2026-08-09 00:41:50 on dist-test-slave-1zqn"
server_key: "f4a32d6808f09c45cf545d597b118148"
server_key_iv: "3943ecbc055d2e7e94d8606dc909ed79"
server_key_version: "encryptionkey@0"
I20260809 00:41:50.603145 31764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f8fc780d41494b0bba46d99fc59b40dd"
format_stamp: "Formatted at 2026-08-09 00:41:50 on dist-test-slave-1zqn"
server_key: "f4a32d6808f09c45cf545d597b118148"
server_key_iv: "3943ecbc055d2e7e94d8606dc909ed79"
server_key_version: "encryptionkey@0"
I20260809 00:41:50.604635 31764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 00:41:50.605403 31782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:50.605619 31764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:50.605717 31764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f8fc780d41494b0bba46d99fc59b40dd"
format_stamp: "Formatted at 2026-08-09 00:41:50 on dist-test-slave-1zqn"
server_key: "f4a32d6808f09c45cf545d597b118148"
server_key_iv: "3943ecbc055d2e7e94d8606dc909ed79"
server_key_version: "encryptionkey@0"
I20260809 00:41:50.605796 31764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:50.637192 31764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:50.639343 31764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:50.639463 31764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:50.643905 31764 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:44579
I20260809 00:41:50.643980 31841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:44579 every 8 connection(s)
I20260809 00:41:50.644376 31764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:50.645386 31842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:50.648010 31842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd: Bootstrap starting.
I20260809 00:41:50.648662 31842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:50.648919 31842 log.cc:824] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:50.649711 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31764
I20260809 00:41:50.649806 31842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd: No bootstrap required, opened a new log
I20260809 00:41:50.649837 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:50.650086 7589 external_mini_cluster.cc:1428] Setting key de89074222dab66fe57e7773513bab62
I20260809 00:41:50.651881 31842 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } }
I20260809 00:41:50.652099 31842 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:50.652155 31842 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8fc780d41494b0bba46d99fc59b40dd, State: Initialized, Role: FOLLOWER
I20260809 00:41:50.652329 31842 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } }
I20260809 00:41:50.652421 31842 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:50.652446 31842 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:50.652504 31842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:50.653121 31842 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } }
I20260809 00:41:50.653213 31842 leader_election.cc:302] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8fc780d41494b0bba46d99fc59b40dd; no voters:
I20260809 00:41:50.653406 31842 leader_election.cc:288] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:50.653453 31847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:50.653589 31847 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [term 1 LEADER]: Becoming Leader. State: Replica: f8fc780d41494b0bba46d99fc59b40dd, State: Running, Role: LEADER
I20260809 00:41:50.653684 31847 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } }
I20260809 00:41:50.653707 31842 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:50.653998 31849 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8fc780d41494b0bba46d99fc59b40dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } } }
I20260809 00:41:50.654093 31849 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:50.654259 31850 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8fc780d41494b0bba46d99fc59b40dd. Latest consensus state: current_term: 1 leader_uuid: "f8fc780d41494b0bba46d99fc59b40dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8fc780d41494b0bba46d99fc59b40dd" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 44579 } } }
I20260809 00:41:50.654325 31853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:50.654907 31853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:50.654345 31850 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:50.657189 31853 catalog_manager.cc:1347] Generated new cluster ID: 3bfaa90dcdda4edfa441c09effbbaf7b
I20260809 00:41:50.657248 31853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:50.657509 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:50.650693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55119 (local address 127.7.105.126:44579)
0809 00:41:50.650867 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:50.650876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:50.650906 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:41:50.651859 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:50.651866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:50.651917 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:50.652036 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:50.652335 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:50.653873 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:50.655025 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:50.655723 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:50.655851 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:50.656648 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:50.656652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:50.656652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:50.656895 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:50.656936 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:50.656954 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:50.657042 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:50.657250 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 00:41:50.657282 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260809 00:41:50.658533 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:50.664125 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:50.660127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58505 (local address 127.7.105.126:44579)
0809 00:41:50.660165 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:50.660167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:50.660283 (+ 116us) server_negotiation.cc:407] Connection header received
0809 00:41:50.660314 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:50.660314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:50.660339 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:50.660406 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:50.660533 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:50.661265 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:50.662249 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:50.662811 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:50.662894 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:50.663720 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:50.663721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:50.663721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:50.663727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:50.663744 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:50.663749 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:50.663818 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:50.663995 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 00:41:50.664011 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:41:50.670759 31853 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:50.671469 31853 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:50.684381 31853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8fc780d41494b0bba46d99fc59b40dd: Generated new TSK 0
I20260809 00:41:50.684620 31853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:50.692898 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31764
W20260809 00:41:50.701133 31749 connection.cc:535] client connection to 127.7.105.126:44579 recv error: Network error: failed to read from TLS socket (remote: 127.7.105.126:44579): Connection reset by peer (error 104)
2026-08-09T00:41:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:50 dist-test-slave-1zqn krb5kdc[31876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:50 dist-test-slave-1zqn krb5kdc[31876](info): set up 2 sockets
Aug 09 00:41:50 dist-test-slave-1zqn krb5kdc[31876](info): commencing operation
krb5kdc: starting...
W20260809 00:41:51.791268 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:51 dist-test-slave-1zqn krb5kdc[31876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:51.925228 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46421
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:46421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:52.011308 31890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:52.011565 31890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:52.011607 31890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:52.012884 31890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:52.012956 31890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:52.012984 31890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:52.013013 31890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:52.013036 31890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:52.014676 31890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:46421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:52.014955 31890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:52.015197 31890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:52.020459 31895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:52.020473 31898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:52.020460 31896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:52.020637 31890 server_base.cc:1034] running on GCE node
I20260809 00:41:52.020926 31890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:52.021236 31890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:52.022404 31890 hybrid_clock.cc:648] HybridClock initialized: now 1786236112022394 us; error 48 us; skew 500 ppm
Aug 09 00:41:52 dist-test-slave-1zqn krb5kdc[31876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236112, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:52.025064 31890 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:52.025559 31890 webserver.cc:460] Webserver started at http://127.7.105.126:39995/ using document root <none> and password file <none>
I20260809 00:41:52.025763 31890 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:52.025803 31890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:52.025894 31890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:52.026926 31890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "fda098af5d4c4d0e80050fc69b2068bc"
format_stamp: "Formatted at 2026-08-09 00:41:52 on dist-test-slave-1zqn"
server_key: "dfb7987905564e8210a0009e1c8be695"
server_key_iv: "b53866229f19fe3c827e7705e8f3eb31"
server_key_version: "encryptionkey@0"
I20260809 00:41:52.027243 31890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "fda098af5d4c4d0e80050fc69b2068bc"
format_stamp: "Formatted at 2026-08-09 00:41:52 on dist-test-slave-1zqn"
server_key: "dfb7987905564e8210a0009e1c8be695"
server_key_iv: "b53866229f19fe3c827e7705e8f3eb31"
server_key_version: "encryptionkey@0"
I20260809 00:41:52.028602 31890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:52.029335 31906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:52.029512 31890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:52.029605 31890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "fda098af5d4c4d0e80050fc69b2068bc"
format_stamp: "Formatted at 2026-08-09 00:41:52 on dist-test-slave-1zqn"
server_key: "dfb7987905564e8210a0009e1c8be695"
server_key_iv: "b53866229f19fe3c827e7705e8f3eb31"
server_key_version: "encryptionkey@0"
I20260809 00:41:52.029662 31890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:52.032939 31890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:52.034971 31890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:52.035079 31890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:52.039273 31890 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:46421
I20260809 00:41:52.039335 31962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:46421 every 8 connection(s)
I20260809 00:41:52.039664 31890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:52.040874 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 31890
I20260809 00:41:52.040933 31963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:52.041014 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:52.041239 7589 external_mini_cluster.cc:1428] Setting key f59db2532f7c64a83a8a2ab436a1ccbf
I20260809 00:41:52.044152 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:52.041819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47569 (local address 127.7.105.126:46421)
0809 00:41:52.042013 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:52.042023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:52.042073 (+ 50us) server_negotiation.cc:407] Connection header received
0809 00:41:52.042658 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:52.042668 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:52.042708 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:52.042898 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:52.043636 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:52.043640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:52.043640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:52.043646 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:52.043698 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:52.043714 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:52.043793 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:52.043853 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:41:52.043896 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260809 00:41:52.044790 31963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc: Bootstrap starting.
I20260809 00:41:52.045274 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:52.045639 31963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:52.046010 31963 log.cc:824] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:52.046744 31963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc: No bootstrap required, opened a new log
I20260809 00:41:52.048214 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:52.047040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34549 (local address 127.7.105.126:46421)
0809 00:41:52.047085 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:52.047087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:52.047218 (+ 131us) server_negotiation.cc:407] Connection header received
0809 00:41:52.047248 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:52.047248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:52.047253 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:52.047323 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:52.047895 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:52.047896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:52.047897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:52.047901 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:52.047916 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:52.047920 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:52.047982 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:52.048140 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:41:52.048155 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:41:52.049270 31963 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } }
I20260809 00:41:52.049433 31963 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:52.049481 31963 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda098af5d4c4d0e80050fc69b2068bc, State: Initialized, Role: FOLLOWER
I20260809 00:41:52.049615 31963 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } }
I20260809 00:41:52.049710 31963 raft_consensus.cc:388] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:52.049755 31963 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:52.049805 31963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:52.050374 31963 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } }
I20260809 00:41:52.050500 31963 leader_election.cc:302] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fda098af5d4c4d0e80050fc69b2068bc; no voters:
I20260809 00:41:52.050704 31963 leader_election.cc:288] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:52.050761 31974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:52.050951 31974 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [term 1 LEADER]: Becoming Leader. State: Replica: fda098af5d4c4d0e80050fc69b2068bc, State: Running, Role: LEADER
I20260809 00:41:52.050987 31963 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:52.051102 31974 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } }
I20260809 00:41:52.051512 31974 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda098af5d4c4d0e80050fc69b2068bc. Latest consensus state: current_term: 1 leader_uuid: "fda098af5d4c4d0e80050fc69b2068bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } } }
I20260809 00:41:52.051601 31974 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:52.051551 31975 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fda098af5d4c4d0e80050fc69b2068bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda098af5d4c4d0e80050fc69b2068bc" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46421 } } }
I20260809 00:41:52.051771 31975 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:52.052021 31982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:52.052614 31982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:52.054243 31982 catalog_manager.cc:1347] Generated new cluster ID: e7605e43e46e427092e2c08a8ba1481e
I20260809 00:41:52.054288 31982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:52.061815 31982 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:52.061983 31982 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:52.065043 31982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fda098af5d4c4d0e80050fc69b2068bc: Generated new TSK 0
I20260809 00:41:52.065240 31982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:52.071347 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 31890
2026-08-09T00:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:52 dist-test-slave-1zqn krb5kdc[31998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:52 dist-test-slave-1zqn krb5kdc[31998](info): set up 2 sockets
Aug 09 00:41:52 dist-test-slave-1zqn krb5kdc[31998](info): commencing operation
krb5kdc: starting...
W20260809 00:41:53.175113 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[31998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:53.307068 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46315
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:46315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:53.393584 32011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:53.393834 32011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:53.393877 32011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:53.395045 32011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:53.395125 32011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:53.395150 32011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:53.395179 32011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:53.395210 32011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:53.396803 32011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:46315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:46315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:53.397105 32011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:53.397349 32011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:53.402503 32016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:53.402503 32019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:53.402604 32011 server_base.cc:1034] running on GCE node
W20260809 00:41:53.402508 32017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:53.402974 32011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:53.403242 32011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:53.404383 32011 hybrid_clock.cc:648] HybridClock initialized: now 1786236113404354 us; error 51 us; skew 500 ppm
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[31998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236113, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:53.406936 32011 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:53.407529 32011 webserver.cc:460] Webserver started at http://127.7.105.126:43509/ using document root <none> and password file <none>
I20260809 00:41:53.407788 32011 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:53.407835 32011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:53.408041 32011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:53.409165 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "f1370ad369914932ae97843e579d54bf"
format_stamp: "Formatted at 2026-08-09 00:41:53 on dist-test-slave-1zqn"
server_key: "7e307ee2247e7119980138014c445ac3"
server_key_iv: "9d71a8e5434fd117e0e95dee1fc2c7d9"
server_key_version: "encryptionkey@0"
I20260809 00:41:53.409475 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "f1370ad369914932ae97843e579d54bf"
format_stamp: "Formatted at 2026-08-09 00:41:53 on dist-test-slave-1zqn"
server_key: "7e307ee2247e7119980138014c445ac3"
server_key_iv: "9d71a8e5434fd117e0e95dee1fc2c7d9"
server_key_version: "encryptionkey@0"
I20260809 00:41:53.410799 32011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:41:53.411530 32026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:53.411708 32011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:41:53.411815 32011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "f1370ad369914932ae97843e579d54bf"
format_stamp: "Formatted at 2026-08-09 00:41:53 on dist-test-slave-1zqn"
server_key: "7e307ee2247e7119980138014c445ac3"
server_key_iv: "9d71a8e5434fd117e0e95dee1fc2c7d9"
server_key_version: "encryptionkey@0"
I20260809 00:41:53.411901 32011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:53.423791 32011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:53.425797 32011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:53.425983 32011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:53.430469 32011 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:46315
I20260809 00:41:53.430543 32083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:46315 every 8 connection(s)
I20260809 00:41:53.430917 32011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:53.431838 32084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:53.432919 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32011
I20260809 00:41:53.433046 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:53.433261 7589 external_mini_cluster.cc:1428] Setting key 541a54c80e545b33b22b122b666e70e9
I20260809 00:41:53.435479 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: Bootstrap starting.
I20260809 00:41:53.436188 32084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:53.436493 32084 log.cc:824] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:53.437168 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: No bootstrap required, opened a new log
I20260809 00:41:53.439050 32084 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } }
I20260809 00:41:53.439237 32084 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:53.439286 32084 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1370ad369914932ae97843e579d54bf, State: Initialized, Role: FOLLOWER
I20260809 00:41:53.439429 32084 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } }
I20260809 00:41:53.439532 32084 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:53.439572 32084 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:53.439638 32084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 0 FOLLOWER]: Advancing to term 1
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[31998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:53.440460 32084 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } }
I20260809 00:41:53.440579 32084 leader_election.cc:302] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1370ad369914932ae97843e579d54bf; no voters:
I20260809 00:41:53.440836 32084 leader_election.cc:288] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:53.440908 32089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:53.441083 32089 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [term 1 LEADER]: Becoming Leader. State: Replica: f1370ad369914932ae97843e579d54bf, State: Running, Role: LEADER
I20260809 00:41:53.441129 32084 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:53.441188 32089 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } }
I20260809 00:41:53.441562 32089 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1370ad369914932ae97843e579d54bf. Latest consensus state: current_term: 1 leader_uuid: "f1370ad369914932ae97843e579d54bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } } }
I20260809 00:41:53.441648 32089 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:53.441584 32090 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1370ad369914932ae97843e579d54bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1370ad369914932ae97843e579d54bf" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 46315 } } }
I20260809 00:41:53.441814 32090 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf [sys.catalog]: This master's current role is: LEADER
W20260809 00:41:53.442961 32106 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:53.443032 32106 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:53.443634 32097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:53.443840 32097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:53.444986 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:53.433816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46189 (local address 127.7.105.126:46315)
0809 00:41:53.433996 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:53.434005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:53.434031 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:41:53.434469 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:53.434475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:53.434520 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:53.434644 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:53.435478 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:53.436526 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:53.437352 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:53.437768 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:53.437898 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:53.440747 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:53.440752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:53.440752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:53.440771 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:53.442841 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:53.443255 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:53.443258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:53.443259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:53.443371 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:53.443662 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:53.443663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:53.443664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:53.443979 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:53.444151 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:53.444635 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 00:41:53.444780 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260809 00:41:53.446115 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:53.452325 32097 catalog_manager.cc:1347] Generated new cluster ID: f5a521c595c24b82aa4fefcc10cb58c6
I20260809 00:41:53.452415 32097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:53.455055 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:53.447826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42559 (local address 127.7.105.126:46315)
0809 00:41:53.447867 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:53.447869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:53.447963 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:41:53.448022 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:53.448022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:53.448031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:53.448153 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:53.448828 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:53.449312 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:53.450351 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:53.450583 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:53.450633 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:53.452155 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:53.452157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:53.452157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:53.452164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:53.453822 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:53.454222 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:53.454223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:53.454224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:53.454289 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:53.454497 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:53.454498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:53.454498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:53.454652 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:53.454756 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:53.454872 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 00:41:53.454959 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:53.466739 32097 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:53.467551 32097 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:53.470438 32097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1370ad369914932ae97843e579d54bf: Generated new TSK 0
I20260809 00:41:53.470659 32097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:53.474237 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32011
2026-08-09T00:41:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[32117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[32117](info): set up 2 sockets
Aug 09 00:41:53 dist-test-slave-1zqn krb5kdc[32117](info): commencing operation
krb5kdc: starting...
W20260809 00:41:54.633661 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:54 dist-test-slave-1zqn krb5kdc[32117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:54.767858 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41711
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:54.853658 32133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:54.853888 32133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:54.853914 32133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:54.855183 32133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:54.855238 32133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:54.855253 32133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:54.855266 32133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:54.855278 32133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:54.856912 32133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:54.857213 32133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:54.857471 32133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:54.862946 32138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:54.863005 32133 server_base.cc:1034] running on GCE node
W20260809 00:41:54.862923 32141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:54.862923 32139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:54.863562 32133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:54.863955 32133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:54.865126 32133 hybrid_clock.cc:648] HybridClock initialized: now 1786236114865117 us; error 57 us; skew 500 ppm
Aug 09 00:41:54 dist-test-slave-1zqn krb5kdc[32117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236114, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:54.871572 32133 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:54.872117 32133 webserver.cc:460] Webserver started at http://127.7.105.126:34969/ using document root <none> and password file <none>
I20260809 00:41:54.872398 32133 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:54.872473 32133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:54.872633 32133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:54.873754 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "303f9fae65f04470afc72ab5075ea04b"
format_stamp: "Formatted at 2026-08-09 00:41:54 on dist-test-slave-1zqn"
server_key: "13f974e6e11ab5aed7724bb8d0231019"
server_key_iv: "ee0442c9329064274a3e52193e168320"
server_key_version: "encryptionkey@0"
I20260809 00:41:54.874137 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "303f9fae65f04470afc72ab5075ea04b"
format_stamp: "Formatted at 2026-08-09 00:41:54 on dist-test-slave-1zqn"
server_key: "13f974e6e11ab5aed7724bb8d0231019"
server_key_iv: "ee0442c9329064274a3e52193e168320"
server_key_version: "encryptionkey@0"
I20260809 00:41:54.875507 32133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:54.876358 32148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:54.876538 32133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:54.876659 32133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "303f9fae65f04470afc72ab5075ea04b"
format_stamp: "Formatted at 2026-08-09 00:41:54 on dist-test-slave-1zqn"
server_key: "13f974e6e11ab5aed7724bb8d0231019"
server_key_iv: "ee0442c9329064274a3e52193e168320"
server_key_version: "encryptionkey@0"
I20260809 00:41:54.876742 32133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:54.897382 32133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:54.899922 32133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:54.900142 32133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:54.904637 32133 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:41711
I20260809 00:41:54.904697 32204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:41711 every 8 connection(s)
I20260809 00:41:54.905088 32133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:54.906041 32205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:54.909173 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b: Bootstrap starting.
I20260809 00:41:54.909883 32205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:54.910224 32205 log.cc:824] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:54.911031 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b: No bootstrap required, opened a new log
I20260809 00:41:54.912873 32205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } }
I20260809 00:41:54.913017 32205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:54.913039 32205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303f9fae65f04470afc72ab5075ea04b, State: Initialized, Role: FOLLOWER
I20260809 00:41:54.913204 32205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } }
I20260809 00:41:54.913269 32205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:54.913323 32205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:54.913378 32205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:54.914265 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32133
I20260809 00:41:54.914412 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:54.914625 7589 external_mini_cluster.cc:1428] Setting key 39d35ecccb309f84fd586192fa093a33
I20260809 00:41:54.914718 32205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } }
I20260809 00:41:54.914852 32205 leader_election.cc:302] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 303f9fae65f04470afc72ab5075ea04b; no voters:
I20260809 00:41:54.915026 32205 leader_election.cc:288] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:54.915117 32208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:54.915285 32208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [term 1 LEADER]: Becoming Leader. State: Replica: 303f9fae65f04470afc72ab5075ea04b, State: Running, Role: LEADER
I20260809 00:41:54.915469 32208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } }
I20260809 00:41:54.915319 32205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:54.915838 32211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 303f9fae65f04470afc72ab5075ea04b. Latest consensus state: current_term: 1 leader_uuid: "303f9fae65f04470afc72ab5075ea04b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } } }
I20260809 00:41:54.915977 32211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:54.916214 32214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:54.916860 32214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:54.917390 32209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "303f9fae65f04470afc72ab5075ea04b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303f9fae65f04470afc72ab5075ea04b" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41711 } } }
I20260809 00:41:54.917531 32209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:54.918764 32214 catalog_manager.cc:1347] Generated new cluster ID: c7c2bd9d9a7b42e99a5d37d40836d887
I20260809 00:41:54.918816 32214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:54.921432 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:54.915228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41843 (local address 127.7.105.126:41711)
0809 00:41:54.915480 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:54.915490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:54.915531 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:41:54.915657 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:54.915664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:54.915725 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:54.915899 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:54.916210 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:54.917962 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:54.919131 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:54.919739 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:54.919841 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:54.920639 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:54.920643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:54.920644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:54.920848 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:54.920886 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:54.920900 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:54.920964 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:54.921072 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:41:54.921218 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260809 00:41:54.922475 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:54.926508 32214 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:54.926976 32214 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:54.928077 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:54.924079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37563 (local address 127.7.105.126:41711)
0809 00:41:54.924120 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:54.924122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:54.924334 (+ 212us) server_negotiation.cc:407] Connection header received
0809 00:41:54.924401 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:54.924401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:54.924412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:54.924459 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:54.924617 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:54.925643 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:54.926636 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:54.926940 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:54.926989 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:54.927771 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:54.927773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:54.927773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:54.927778 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:54.927793 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:54.927797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:54.927850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:54.927944 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:41:54.927983 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:54.933092 32214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 303f9fae65f04470afc72ab5075ea04b: Generated new TSK 0
I20260809 00:41:54.933300 32214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:54.936486 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32133
2026-08-09T00:41:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:54 dist-test-slave-1zqn krb5kdc[32237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:54 dist-test-slave-1zqn krb5kdc[32237](info): set up 2 sockets
Aug 09 00:41:54 dist-test-slave-1zqn krb5kdc[32237](info): commencing operation
krb5kdc: starting...
W20260809 00:41:56.061080 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:56 dist-test-slave-1zqn krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:56.191351 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34695
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:34695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:56.278200 32250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:56.278438 32250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:56.278461 32250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:56.279583 32250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:56.279624 32250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:56.279639 32250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:56.279651 32250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:56.279663 32250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:56.281101 32250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:34695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:34695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:56.281301 32250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:56.281471 32250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:56.286578 32259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:56.286638 32256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:56.286711 32257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:56.286846 32250 server_base.cc:1034] running on GCE node
I20260809 00:41:56.287235 32250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:56.287576 32250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:56.288730 32250 hybrid_clock.cc:648] HybridClock initialized: now 1786236116288690 us; error 54 us; skew 500 ppm
Aug 09 00:41:56 dist-test-slave-1zqn krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236116, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:56.291311 32250 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:56.291754 32250 webserver.cc:460] Webserver started at http://127.7.105.126:32931/ using document root <none> and password file <none>
I20260809 00:41:56.292105 32250 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:56.292191 32250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:56.292351 32250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:56.293478 32250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "0c7235dbf79d415ea0a118341441d94f"
format_stamp: "Formatted at 2026-08-09 00:41:56 on dist-test-slave-1zqn"
server_key: "c955bd0f14438fae2378dfba7f66bb9f"
server_key_iv: "0a36f8e319a32469bdae278959a38a9f"
server_key_version: "encryptionkey@0"
I20260809 00:41:56.293859 32250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "0c7235dbf79d415ea0a118341441d94f"
format_stamp: "Formatted at 2026-08-09 00:41:56 on dist-test-slave-1zqn"
server_key: "c955bd0f14438fae2378dfba7f66bb9f"
server_key_iv: "0a36f8e319a32469bdae278959a38a9f"
server_key_version: "encryptionkey@0"
I20260809 00:41:56.295241 32250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:56.295965 32266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:56.296182 32250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:56.296298 32250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "0c7235dbf79d415ea0a118341441d94f"
format_stamp: "Formatted at 2026-08-09 00:41:56 on dist-test-slave-1zqn"
server_key: "c955bd0f14438fae2378dfba7f66bb9f"
server_key_iv: "0a36f8e319a32469bdae278959a38a9f"
server_key_version: "encryptionkey@0"
I20260809 00:41:56.296375 32250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:56.299775 32250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:56.302145 32250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:56.302287 32250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:56.306368 32250 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:34695
I20260809 00:41:56.306404 32323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:34695 every 8 connection(s)
I20260809 00:41:56.306768 32250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:56.307080 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32250
I20260809 00:41:56.307209 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:56.307443 7589 external_mini_cluster.cc:1428] Setting key e37f97253e69a5840952f590554c91b5
I20260809 00:41:56.308189 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:56.310024 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:56.308021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35779 (local address 127.7.105.126:34695)
0809 00:41:56.308227 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:56.308252 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:56.308289 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:41:56.308744 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:56.308750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:56.308774 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:56.308887 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:56.309539 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:56.309542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:56.309543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:56.309547 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:56.309585 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:56.309613 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:56.309670 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:56.309769 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:41:56.309820 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260809 00:41:56.311074 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:56.311636 32324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f: Bootstrap starting.
I20260809 00:41:56.312486 32324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:56.312865 32324 log.cc:824] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:56.313759 32324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f: No bootstrap required, opened a new log
I20260809 00:41:56.313880 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:56.312731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49349 (local address 127.7.105.126:34695)
0809 00:41:56.312772 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:56.312774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:56.312883 (+ 109us) server_negotiation.cc:407] Connection header received
0809 00:41:56.312907 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:56.312907 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:56.312912 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:56.312978 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:56.313600 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:56.313601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:56.313602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:41:56.313606 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:56.313618 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:41:56.313622 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:56.313667 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:56.313749 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 00:41:56.313760 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:56.315929 32324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } }
I20260809 00:41:56.316118 32324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:41:56.316170 32324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7235dbf79d415ea0a118341441d94f, State: Initialized, Role: FOLLOWER
I20260809 00:41:56.316299 32324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } }
I20260809 00:41:56.316385 32324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:41:56.316428 32324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:41:56.316478 32324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:56.317018 32324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } }
I20260809 00:41:56.317138 32324 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7235dbf79d415ea0a118341441d94f; no voters:
I20260809 00:41:56.317335 32324 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:41:56.317453 32334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:56.317633 32334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [term 1 LEADER]: Becoming Leader. State: Replica: 0c7235dbf79d415ea0a118341441d94f, State: Running, Role: LEADER
I20260809 00:41:56.317782 32334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } }
I20260809 00:41:56.317665 32324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:56.318298 32336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7235dbf79d415ea0a118341441d94f. Latest consensus state: current_term: 1 leader_uuid: "0c7235dbf79d415ea0a118341441d94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } } }
I20260809 00:41:56.318409 32336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:56.318662 32338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:56.318650 32335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7235dbf79d415ea0a118341441d94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7235dbf79d415ea0a118341441d94f" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 34695 } } }
I20260809 00:41:56.319151 32335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:56.319238 32338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:56.320993 32338 catalog_manager.cc:1347] Generated new cluster ID: 836866b9b1fd464cb99a758b1dc31d37
I20260809 00:41:56.321075 32338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:41:56.344107 32338 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:56.344345 32338 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:56.361109 32338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7235dbf79d415ea0a118341441d94f: Generated new TSK 0
I20260809 00:41:56.361385 32338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:56.373369 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32250
2026-08-09T00:41:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1434 ms)
[----------] 18 tests from AuthTokenIssuingTest (25571 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:56 dist-test-slave-1zqn krb5kdc[32358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:56 dist-test-slave-1zqn krb5kdc[32358](info): set up 2 sockets
Aug 09 00:41:56 dist-test-slave-1zqn krb5kdc[32358](info): commencing operation
krb5kdc: starting...
W20260809 00:41:57.531344 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:41:57 dist-test-slave-1zqn krb5kdc[32358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:41:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:41:57.660482 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41879
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:41:57.745219 32372 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:41:57.745486 32372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:57.745518 32372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:57.745563 32372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:57.746731 32372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:57.746802 32372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:57.746827 32372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:57.746861 32372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:57.746888 32372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:57.748524 32372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:57.748833 32372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:57.749081 32372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:57.754325 32378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:57.754395 32377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:57.754523 32372 server_base.cc:1034] running on GCE node
W20260809 00:41:57.754678 32380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:57.754956 32372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:57.755287 32372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:57.756457 32372 hybrid_clock.cc:648] HybridClock initialized: now 1786236117756445 us; error 48 us; skew 500 ppm
Aug 09 00:41:57 dist-test-slave-1zqn krb5kdc[32358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:57.759020 32372 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:41:57.759486 32372 webserver.cc:460] Webserver started at http://127.7.105.126:35159/ using document root <none> and password file <none>
I20260809 00:41:57.759752 32372 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:57.759824 32372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:57.759984 32372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:57.761111 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "8e9bb111e6264c54a6e3e4f6dea84200"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "651063c5104a0ea2b7b4eef43cad7d5c"
server_key_iv: "555cf49f70b03b31a74d11ff3c48994f"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.761490 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "8e9bb111e6264c54a6e3e4f6dea84200"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "651063c5104a0ea2b7b4eef43cad7d5c"
server_key_iv: "555cf49f70b03b31a74d11ff3c48994f"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.762907 32372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:41:57.763720 32388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:57.763939 32372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:57.764077 32372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "8e9bb111e6264c54a6e3e4f6dea84200"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "651063c5104a0ea2b7b4eef43cad7d5c"
server_key_iv: "555cf49f70b03b31a74d11ff3c48994f"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.764196 32372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:57.813567 32372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:57.815703 32372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:57.815896 32372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:57.820611 32372 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:41879
I20260809 00:41:57.820658 32444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:41879 every 8 connection(s)
I20260809 00:41:57.821080 32372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:41:57.822180 32445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:57.823828 32445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:57.827275 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32372
I20260809 00:41:57.827381 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:41:57.827601 7589 external_mini_cluster.cc:1428] Setting key 4f3a49ef3a6024889d9ec4de16875776
Aug 09 00:41:57 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:57.834764 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.824788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.7.105.126:41879)
0809 00:41:57.824924 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:57.824928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:57.824959 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:41:57.825486 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:57.825488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:57.825504 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:57.825584 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:57.826151 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.826998 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.828145 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.828457 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.828608 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:57.831215 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:57.831223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:57.831223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:57.831230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:57.833015 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:57.833531 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:57.833533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:57.833534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:57.833620 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:57.833905 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:57.833906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:57.833906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:57.834144 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:57.834210 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:57.834477 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 00:41:57.834583 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260809 00:41:57.834861 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.824612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41879 (local address 127.0.0.1:41838)
0809 00:41:57.824797 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.824840 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:57.825348 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:57.825615 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:57.825617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:57.825875 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:57.826041 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:57.826045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.827113 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.827114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:57.827996 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:57.827999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.828583 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.828585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:57.828651 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:57.829289 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:57.829298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:57.831114 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:57.833105 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:57.833114 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:57.833117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:57.833432 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:57.833692 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:57.833694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:57.833694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:57.833831 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:57.834221 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:57.834230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:57.834320 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 00:41:57.834441 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:41:57.834694 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":71,"thread_start_us":80,"threads_started":1}
I20260809 00:41:57.835705 32445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:57.836133 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.835955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35132)
0809 00:41:57.836015 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.836098 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:41:57.836292 32392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
I20260809 00:41:57.836793 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.836663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35146)
0809 00:41:57.836725 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.836759 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 00:41:57.836997 32445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:36149: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.105.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
I20260809 00:41:57.857208 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:36149
--webserver_interface=127.7.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:41:57.886235 32445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } attempt: 1
I20260809 00:41:57.886826 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.886655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35150)
0809 00:41:57.886735 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.886773 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 00:41:57.887204 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.887102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35162)
0809 00:41:57.887150 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.887176 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:41:57.887298 32445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:36149: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
W20260809 00:41:57.946601 32452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:41:57.946892 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:57.946925 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:57.946974 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:57.948345 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:57.948412 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:57.948441 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:57.948463 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:57.948485 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:57.950260 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:36149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:57.950584 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:57.950843 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:41:57.955461 32445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } attempt: 2
I20260809 00:41:57.955974 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.955743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35170)
0809 00:41:57.955833 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.955881 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 00:41:57.956257 32459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:57.956282 32458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:57.956470 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.956361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35182)
0809 00:41:57.956416 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.956443 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:41:57.956589 32445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:36149: Network error: Client connection negotiation failed: client connection to 127.7.105.125:36149: connect: Connection refused (error 111)
W20260809 00:41:57.956585 32461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:57.956619 32452 server_base.cc:1034] running on GCE node
I20260809 00:41:57.956853 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:57.957116 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:57.958302 32452 hybrid_clock.cc:648] HybridClock initialized: now 1786236117958292 us; error 41 us; skew 500 ppm
Aug 09 00:41:57 dist-test-slave-1zqn krb5kdc[32358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:57.961117 32452 init.cc:377] Logged in from keytab as kudu/127.7.105.125@KRBTEST.COM (short username kudu)
I20260809 00:41:57.961576 32452 webserver.cc:460] Webserver started at http://127.7.105.125:41003/ using document root <none> and password file <none>
I20260809 00:41:57.961823 32452 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:57.961894 32452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:57.962044 32452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:57.963204 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data/instance:
uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "0609e94410364357345d256d70c2e651"
server_key_iv: "83eedac0b14f79533e7db20c6ae50dee"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.963589 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal/instance:
uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "0609e94410364357345d256d70c2e651"
server_key_iv: "83eedac0b14f79533e7db20c6ae50dee"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.965288 32452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:41:57.966264 32470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:57.966553 32452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:41:57.966671 32452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87"
format_stamp: "Formatted at 2026-08-09 00:41:57 on dist-test-slave-1zqn"
server_key: "0609e94410364357345d256d70c2e651"
server_key_iv: "83eedac0b14f79533e7db20c6ae50dee"
server_key_version: "encryptionkey@0"
I20260809 00:41:57.966756 32452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:57.982617 32452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:57.985230 32452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:57.985407 32452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:57.989686 32452 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.125:36149
I20260809 00:41:57.989729 32526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.125:36149 every 8 connection(s)
I20260809 00:41:57.990123 32452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
I20260809 00:41:57.991225 32527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:57.993223 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32452
I20260809 00:41:57.993333 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-1/wal/instance
I20260809 00:41:57.993074 32527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:57.993507 7589 external_mini_cluster.cc:1428] Setting key 2c23c36e3a1c697d1e770f475ae8cc7b
Aug 09 00:41:57 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.124@KRBTEST.COM; defaulting to no policy
I20260809 00:41:58.003026 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.993955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.7.105.126:41879)
0809 00:41:57.994026 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:57.994029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:57.994269 (+ 240us) server_negotiation.cc:407] Connection header received
0809 00:41:57.994732 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:57.994734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:57.994749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:57.994829 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:57.995471 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.995977 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.997022 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.997242 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.997332 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:57.999989 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:57.999991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:57.999991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:57.999999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.001508 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.002025 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.002026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.002026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.002080 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.002361 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.002361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.002362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.002464 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.002546 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.002837 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.002926 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:41:58.003276 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:57.993939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41879 (local address 127.0.0.1:41842)
0809 00:41:57.994131 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:57.994174 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:57.994641 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:57.994884 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:57.994887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:57.995167 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:57.995364 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:57.995368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.996111 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.996112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:57.996926 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:57.996929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:57.997357 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:57.997358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:57.997529 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:57.998115 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:57.998123 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:57.999889 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.001599 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.001609 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.001611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.001944 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.002149 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.002149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.002150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.002295 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.002540 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.002549 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.002697 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.002813 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.002985 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260809 00:41:58.003893 32527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.124@KRBTEST.COM" created.
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:41:58.013046 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.004193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35190)
0809 00:41:58.004251 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.004265 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.004310 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.004635 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.004636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.004822 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.004879 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.004881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.006012 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.006013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.006599 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.006601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.007281 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.007282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.007343 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.007865 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.007872 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.009448 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.011848 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.011849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.011849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.012025 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.012333 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.012333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.012334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.012390 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.012771 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.012772 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.012811 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.012868 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.012962 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:41:58.013195 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.004307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.7.105.125:36149)
0809 00:41:58.004463 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.004466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.004482 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:41:58.004499 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.004500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.004511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.004588 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.004958 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.005844 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.006690 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.007195 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.007272 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.009553 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.009563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.009564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.009572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.011738 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.012177 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.012182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.012182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.012254 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.012468 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.012469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.012469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.012693 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.012795 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.012908 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.013035 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 00:41:58.013842 32527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:58.014264 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.014128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42176)
0809 00:41:58.014185 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.014243 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:41:58.014436 32472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
I20260809 00:41:58.014906 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.014810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42186)
0809 00:41:58.014859 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.014886 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:41:58.015053 32527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33729: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
I20260809 00:41:58.026819 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:33729
--webserver_interface=127.7.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:43999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:41:58.047772 32445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } attempt: 3
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:41:58.055195 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.048343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.7.105.125:36149)
0809 00:41:58.048419 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.048421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.048434 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:41:58.048503 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.048504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.048516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.048615 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.048945 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.049444 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.050142 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.050370 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.050450 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.052690 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.052691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.052691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.052699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.054073 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.054465 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.054466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.054466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.054519 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.054720 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.054721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.054721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.054831 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.054888 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.055055 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.055124 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:41:58.055239 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.048261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35202)
0809 00:41:58.048338 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.048358 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.048434 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.048629 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.048630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.048796 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.048862 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.048863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.049540 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.049546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.050049 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.050058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.050446 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.050447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.050533 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.051012 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.051016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.052598 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.054162 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.054163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.054163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.054390 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.054585 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.054586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.054586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.054657 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.054900 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.054901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.054955 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.055027 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.055132 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:41:58.055794 32445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:58.056183 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.056041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42198)
0809 00:41:58.056113 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.056149 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 00:41:58.056486 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.056424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42212)
0809 00:41:58.056453 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.056473 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 00:41:58.056588 32445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33729: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
I20260809 00:41:58.064301 32527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } attempt: 1
I20260809 00:41:58.064806 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.064663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42222)
0809 00:41:58.064729 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.064773 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:41:58.065210 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.065114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42232)
0809 00:41:58.065162 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.065190 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:41:58.065384 32527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33729: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
I20260809 00:41:58.087745 32445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } attempt: 1
I20260809 00:41:58.088303 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.088163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42234)
0809 00:41:58.088228 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.088272 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 00:41:58.088667 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.088565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42236)
0809 00:41:58.088612 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.088648 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:41:58.088766 32445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33729: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
W20260809 00:41:58.114830 32534 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:41:58.115098 32534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:41:58.115129 32534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:41:58.115172 32534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:41:58.116343 32534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:41:58.116412 32534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:41:58.116438 32534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:41:58.116460 32534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:41:58.116489 32534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:41:58.118000 32534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:43999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:41879,127.7.105.125:36149,127.7.105.124:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:41:58.118287 32534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:41:58.118523 32534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:41:58.123864 32540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:41:58.123873 32542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:58.124082 32534 server_base.cc:1034] running on GCE node
W20260809 00:41:58.124195 32539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:41:58.124470 32534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:41:58.124776 32534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:41:58.125953 32534 hybrid_clock.cc:648] HybridClock initialized: now 1786236118125937 us; error 44 us; skew 500 ppm
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236118, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:41:58.128573 32534 init.cc:377] Logged in from keytab as kudu/127.7.105.124@KRBTEST.COM (short username kudu)
I20260809 00:41:58.129035 32534 webserver.cc:460] Webserver started at http://127.7.105.124:42171/ using document root <none> and password file <none>
I20260809 00:41:58.129266 32534 fs_manager.cc:359] Metadata directory not provided
I20260809 00:41:58.129336 32534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:41:58.129490 32534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:41:58.130554 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data/instance:
uuid: "e8b5eff5baa94b74895fbd61e0ca6bde"
format_stamp: "Formatted at 2026-08-09 00:41:58 on dist-test-slave-1zqn"
server_key: "b25915360b79e70718f046ea7993fae2"
server_key_iv: "738b7c98bb8ed87f2fd117325462ceda"
server_key_version: "encryptionkey@0"
I20260809 00:41:58.130936 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal/instance:
uuid: "e8b5eff5baa94b74895fbd61e0ca6bde"
format_stamp: "Formatted at 2026-08-09 00:41:58 on dist-test-slave-1zqn"
server_key: "b25915360b79e70718f046ea7993fae2"
server_key_iv: "738b7c98bb8ed87f2fd117325462ceda"
server_key_version: "encryptionkey@0"
I20260809 00:41:58.132349 32534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:41:58.133131 32549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:41:58.133303 32534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:41:58.133414 32534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "e8b5eff5baa94b74895fbd61e0ca6bde"
format_stamp: "Formatted at 2026-08-09 00:41:58 on dist-test-slave-1zqn"
server_key: "b25915360b79e70718f046ea7993fae2"
server_key_iv: "738b7c98bb8ed87f2fd117325462ceda"
server_key_version: "encryptionkey@0"
I20260809 00:41:58.133508 32534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:41:58.133548 32527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } attempt: 2
I20260809 00:41:58.134038 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.133821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42246)
0809 00:41:58.133969 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.134008 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260809 00:41:58.134419 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.134322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42250)
0809 00:41:58.134368 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.134393 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:41:58.134508 32527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:33729: Network error: Client connection negotiation failed: client connection to 127.7.105.124:33729: connect: Connection refused (error 111)
I20260809 00:41:58.145509 32534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:41:58.147873 32534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:41:58.148046 32534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:41:58.152490 32534 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.124:33729
I20260809 00:41:58.152527 32607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.124:33729 every 8 connection(s)
I20260809 00:41:58.152915 32534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
I20260809 00:41:58.153954 32608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:41:58.155591 32608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236118, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:58.163161 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 32534
I20260809 00:41:58.163295 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235843038573-7589-0/minicluster-data/master-2/wal/instance
I20260809 00:41:58.163534 7589 external_mini_cluster.cc:1428] Setting key 98733f1c2153cd2d32da6cc053b9d0c8
I20260809 00:41:58.163905 32445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } attempt: 2
I20260809 00:41:58.165583 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.156426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.7.105.126:41879)
0809 00:41:58.156504 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.156506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.156726 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:41:58.157184 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.157184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.157194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.157258 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.157866 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.158329 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.159342 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.159532 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.159615 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.162126 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.162127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.162128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.162135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.163972 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.164559 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.164560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.164560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.164627 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.164895 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.164896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.164897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.165032 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.165111 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.165405 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.165488 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:41:58.165822 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.156407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:41879 (local address 127.0.0.1:41844)
0809 00:41:58.156588 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.156646 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.157097 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.157313 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.157317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.157576 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.157789 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.157793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.158417 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.158418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.159247 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.159249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.159663 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.159664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.159780 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.160454 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.160463 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.162012 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.164097 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.164107 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.164109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.164461 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.164698 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.164699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.164699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.164814 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.165102 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.165109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.165262 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.165371 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.165552 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260809 00:41:58.167011 32608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236118, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:41:58.175359 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.164234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42278)
0809 00:41:58.164332 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.164343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.164381 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.164729 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.164731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.164905 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.164965 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.164967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.166396 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.166398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.167853 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.167857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.168321 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.168322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.168380 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.168901 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.168906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.170714 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.172733 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.172734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.172735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.172960 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.174365 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.174366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.174452 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.175004 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.175005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.175054 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.175145 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.175257 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 00:41:58.175374 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.164381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.7.105.124:33729)
0809 00:41:58.164534 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.164539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.164563 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:41:58.164588 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.164590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.164606 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.165051 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.165118 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.166271 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.167951 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.168262 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.168319 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.170797 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.170805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.170806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.170812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.172650 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.173399 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.173404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.173405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.174285 (+ 880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174683 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.174685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.174685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.174926 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.175025 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.175183 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.175259 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260809 00:41:58.175498 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.167444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.7.105.125:36149)
0809 00:41:58.167506 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.167508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.167517 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:58.167546 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.167547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.167559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.167624 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.167967 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.168701 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.169807 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.170110 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.170148 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.172404 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.172405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.172406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.172417 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.174163 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174534 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.174535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.174535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.174595 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174866 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.174868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.174868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.175024 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.175075 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.175296 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.175392 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:41:58.175832 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.164217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.7.105.124:33729)
0809 00:41:58.164534 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.164539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.164563 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:41:58.164588 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.164590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.164609 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.165464 (+ 855us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.165494 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.166286 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.167355 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.168033 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.168172 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.171597 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.171598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.171598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.171606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.174046 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174473 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.174474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.174475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.174533 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174748 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.174749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.174749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.174962 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.175548 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.175594 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.175683 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":282,"thread_start_us":101,"threads_started":1}
I20260809 00:41:58.175843 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.167339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:36149 (local address 127.0.0.1:35208)
0809 00:41:58.167396 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.167406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.167484 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.167628 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.167629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.167781 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.167891 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.167893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.168811 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.168813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.169694 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.169698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.170257 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.170257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.170332 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.170766 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.170772 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.172297 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.174249 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.174250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.174454 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.174678 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.174678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.174679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.174767 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.175131 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.175132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.175170 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.175581 (+ 411us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.175738 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":67}
I20260809 00:41:58.176537 32608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:41:58.177873 32445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: Bootstrap starting.
I20260809 00:41:58.178695 32445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:58.179111 32445 log.cc:824] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:58.180580 32445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: No bootstrap required, opened a new log
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236118, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:41:58.182811 32445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.183019 32445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9bb111e6264c54a6e3e4f6dea84200, State: Initialized, Role: FOLLOWER
I20260809 00:41:58.183164 32445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.183617 32445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:58.184357 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.176818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42282)
0809 00:41:58.176872 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.176883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.176925 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.177088 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.177089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.177244 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.177281 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.177283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.178051 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.178052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.178658 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.178660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.179244 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.179245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.179300 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.179755 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.179759 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.181202 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.183230 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.183231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.183231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.183437 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.183652 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.183652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.183653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.183714 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.183983 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.183984 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.184022 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.184148 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.184240 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 00:41:58.184648 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.176872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.7.105.124:33729)
0809 00:41:58.176915 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.176917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.176927 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:41:58.176987 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.176988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.176999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.177052 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.177373 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.177986 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.178756 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.179113 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.179194 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.181297 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.181298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.181298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.181304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.183147 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.183514 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.183515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.183516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.183571 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.183765 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.183766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.183767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.183908 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.183981 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.184482 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.184553 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:41:58.185572 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.176702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.7.105.126:41879)
0809 00:41:58.176744 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.176746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.176960 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:41:58.177048 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.177049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.177059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.177104 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.177664 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.178321 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.179172 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.179386 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.179449 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.182061 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.182062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.182062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.182070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.184027 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.184427 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.184428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.184428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.184495 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.184773 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.184775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.184775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.184937 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.185105 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.185381 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.185477 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:41:58.185804 32627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:58.185889 32627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:58.186229 32608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: Bootstrap starting.
I20260809 00:41:58.186293 32616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.186396 32616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.186971 32608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:58.187212 32608 log.cc:824] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:58.187911 32608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: No bootstrap required, opened a new log
I20260809 00:41:58.189433 32608 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.189580 32608 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b5eff5baa94b74895fbd61e0ca6bde, State: Initialized, Role: FOLLOWER
I20260809 00:41:58.189654 32608 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.189976 32608 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:58.186759 32372 master_runner.cc:186] Error getting master registration for 127.7.105.125:36149: OK, instance_id { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" instance_seqno: 1786236117988412 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:41:58.189936 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.190030 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.190045 32372 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:41:58.196628 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.186584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.7.105.125:36149)
0809 00:41:58.186639 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.186641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.186650 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:58.186675 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.186676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.186687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.186743 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.187552 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.188418 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.189381 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.189682 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.189763 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.192835 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.192836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.192837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.192845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.194765 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.195283 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.195285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.195285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.195354 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.195630 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.195631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.195631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.195774 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.195928 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.196447 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.196532 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:41:58.194265 32534 master_runner.cc:186] Error getting master registration for 127.7.105.125:36149: OK, instance_id { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" instance_seqno: 1786236117988412 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:41:58.197196 32534 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 00:41:58.192116 32642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:41:58.197369 32642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:41:58.198408 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:41:58.199470 32418 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8e9bb111e6264c54a6e3e4f6dea84200"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41850
I20260809 00:41:58.199615 32418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:41:58.199674 32418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:58.200711 32418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.200978 32418 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [CANDIDATE]: Term 1 election: Requested vote from peers 9fec51f8c52b4272a4e9bcff3c4fdd87 (127.7.105.125:36149), e8b5eff5baa94b74895fbd61e0ca6bde (127.7.105.124:33729)
I20260809 00:41:58.201340 32580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde"
I20260809 00:41:58.201448 32580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:58.202759 32580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9bb111e6264c54a6e3e4f6dea84200 in term 1.
I20260809 00:41:58.202956 32390 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9bb111e6264c54a6e3e4f6dea84200, e8b5eff5baa94b74895fbd61e0ca6bde; no voters:
I20260809 00:41:58.203136 32616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:41:58.203334 32616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9bb111e6264c54a6e3e4f6dea84200, State: Running, Role: LEADER
I20260809 00:41:58.203436 32616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.203909 32616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9bb111e6264c54a6e3e4f6dea84200. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.203984 32616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:58.204195 32650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:41:58.204408 32650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:41:58.206875 32580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [term 1 FOLLOWER]: Refusing update from remote peer 8e9bb111e6264c54a6e3e4f6dea84200: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:58.207556 32616 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:58.208907 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9bb111e6264c54a6e3e4f6dea84200. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.209018 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.217407 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.204448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.7.105.125:36149)
0809 00:41:58.204492 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.204494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.205987 (+ 1493us) server_negotiation.cc:407] Connection header received
0809 00:41:58.206158 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.206159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.206182 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.206218 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.207875 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.208654 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.210497 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.210817 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.210865 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.213543 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.213556 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.213557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.213565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.215503 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.216190 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.216191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.216191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.216266 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.216604 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.216605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.216605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.216739 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.216816 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.217207 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.217292 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:58.224095 32616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.224212 32616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:58.224735 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.224829 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.225077 32652 mvcc.cc:205] Tried to move back new op lower bound from 7316423140171919360 to 7316423140162252800. Current Snapshot: MvccSnapshot[applied={T|T < 7316423140171919360}]
I20260809 00:41:58.225630 32527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } attempt: 3
I20260809 00:41:58.226693 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.205580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.7.105.124:33729)
0809 00:41:58.205640 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.205642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.209826 (+ 4184us) server_negotiation.cc:407] Connection header received
0809 00:41:58.210141 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.210142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.210164 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.210194 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.213192 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.213926 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.217551 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.217868 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.217910 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.222412 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.222413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.222413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.222421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.224165 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.225016 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.225018 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.225018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.225084 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.225586 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.225587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.225587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.225727 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.225847 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.226334 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.226460 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:41:58.226461 32655 mvcc.cc:205] Tried to move back new op lower bound from 7316423140171919360 to 7316423140162252800. Current Snapshot: MvccSnapshot[applied={T|T < 7316423140171919360}]
I20260809 00:41:58.227245 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.205079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.7.105.126:41879)
0809 00:41:58.205122 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.205124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.206348 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 00:41:58.209294 (+ 2946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.209295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.209306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.209335 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.211655 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.212405 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.216264 (+ 3859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.216581 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.216649 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.220844 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.220846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.220847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.220855 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.223020 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.225755 (+ 2735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.225768 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.225768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.225827 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.226113 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.226114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.226114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.226254 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.226332 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.227026 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.227111 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:58.228839 32650 catalog_manager.cc:1347] Generated new cluster ID: 6213ff08f97940bb9821c5c6d350eeb3
I20260809 00:41:58.228893 32650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:41:58 dist-test-slave-1zqn krb5kdc[32358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
I20260809 00:41:58.236471 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.225903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:33729 (local address 127.0.0.1:42302)
0809 00:41:58.226000 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 00:41:58.226011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:41:58.226056 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:41:58.226333 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:41:58.226333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:41:58.226508 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:41:58.226558 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.226561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.227511 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.227512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.228309 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:41:58.228311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.228882 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.228883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:41:58.228947 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.229412 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:41:58.229418 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:41:58.230966 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:41:58.235282 (+ 4316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.235283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.235284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.235520 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.235726 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:41:58.235727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:41:58.235728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:41:58.235793 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:41:58.236123 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:41:58.236124 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:41:58.236173 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:41:58.236244 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:41:58.236352 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 00:41:58.236920 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.226138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.7.105.124:33729)
0809 00:41:58.226184 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.226186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.226194 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:58.226215 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.226216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.226229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.226283 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.226672 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.227405 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.228405 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.228742 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.228816 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.231054 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:41:58.231055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:41:58.231055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:41:58.231078 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:41:58.235188 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.235591 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.235592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.235593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.235654 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:41:58.235884 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:41:58.235885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:41:58.235885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:41:58.236021 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:41:58.236677 (+ 656us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.236721 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.236808 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:41:58.238611 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: Bootstrap starting.
I20260809 00:41:58.239418 32527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: Neither blocks nor log segments found. Creating new log.
I20260809 00:41:58.239751 32527 log.cc:824] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: Log is configured to *not* fsync() on all Append() calls
I20260809 00:41:58.247462 32650 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:41:58.248154 32650 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:41:58.248751 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: No bootstrap required, opened a new log
I20260809 00:41:58.253248 32527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.253564 32527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec51f8c52b4272a4e9bcff3c4fdd87, State: Initialized, Role: FOLLOWER
I20260809 00:41:58.253724 32527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } }
I20260809 00:41:58.254236 32527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:41:58.254981 32498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:41:58.256235 32498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [term 1 FOLLOWER]: Refusing update from remote peer 8e9bb111e6264c54a6e3e4f6dea84200: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:41:58.256635 32644 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:41:58.256682 32499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87"
I20260809 00:41:58.257689 32499 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9bb111e6264c54a6e3e4f6dea84200 in term 1.
I20260809 00:41:58.258697 32662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.258849 32662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.258953 32662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9bb111e6264c54a6e3e4f6dea84200. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.259020 32662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.259064 32662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.259120 32662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:41:58.262956 32644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9bb111e6264c54a6e3e4f6dea84200" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 41879 } } peers { permanent_uuid: "9fec51f8c52b4272a4e9bcff3c4fdd87" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 36149 } } peers { permanent_uuid: "e8b5eff5baa94b74895fbd61e0ca6bde" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 33729 } } }
I20260809 00:41:58.263082 32644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200 [sys.catalog]: This master's current role is: LEADER
I20260809 00:41:58.264487 32650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9bb111e6264c54a6e3e4f6dea84200: Generated new TSK 0
I20260809 00:41:58.264693 32650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:41:58.268702 32675 mvcc.cc:205] Tried to move back new op lower bound from 7316423140335742976 to 7316423140162252800. Current Snapshot: MvccSnapshot[applied={T|T < 7316423140171919360}]
I20260809 00:41:58.272725 32673 catalog_manager.cc:1259] Loaded cluster ID: 6213ff08f97940bb9821c5c6d350eeb3
I20260809 00:41:58.272791 32673 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: loading cluster ID for follower catalog manager: success
I20260809 00:41:58.273931 32673 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: acquiring CA information for follower catalog manager: success
I20260809 00:41:58.274266 32673 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9fec51f8c52b4272a4e9bcff3c4fdd87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:41:58.284017 32531 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.282296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.7.105.125:36149)
0809 00:41:58.282341 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.282343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.283304 (+ 961us) server_negotiation.cc:407] Connection header received
0809 00:41:58.283414 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.283415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.283426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.283457 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.283928 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35234: BlockingRecv error: recv got EOF from 127.0.0.1:35234 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:41:58.285043 32448 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.282725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.7.105.126:41879)
0809 00:41:58.282765 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.282767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.284506 (+ 1739us) server_negotiation.cc:407] Connection header received
0809 00:41:58.284617 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.284618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.284630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.284659 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.284978 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41864: BlockingRecv error: recv got EOF from 127.0.0.1:41864 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:41:58.285980 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.283031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.7.105.124:33729)
0809 00:41:58.283073 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.283075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.285386 (+ 2311us) server_negotiation.cc:407] Connection header received
0809 00:41:58.285543 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.285545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.285556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.285584 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.285920 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42310: BlockingRecv error: recv got EOF from 127.0.0.1:42310 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:58.292647 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.288801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.7.105.125:36149)
0809 00:41:58.288847 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.288849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.289041 (+ 192us) server_negotiation.cc:407] Connection header received
0809 00:41:58.289154 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.289155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.289199 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.289241 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.289454 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.290297 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.291633 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.291904 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.291950 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.292219 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.292342 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.292397 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.292537 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.292559 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:41:58.296343 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.295443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.7.105.124:33729)
0809 00:41:58.295484 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.295486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.295727 (+ 241us) server_negotiation.cc:407] Connection header received
0809 00:41:58.295864 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.295865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.295889 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.295919 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.296286 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42324: BlockingRecv error: recv got EOF from 127.0.0.1:42324 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:58.302006 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.298747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.7.105.125:36149)
0809 00:41:58.298782 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.298784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.299061 (+ 277us) server_negotiation.cc:407] Connection header received
0809 00:41:58.299161 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.299162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.299172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.299200 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.299372 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.300162 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.301200 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.301475 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.301522 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.301746 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.301781 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.301809 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.301917 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.301936 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 00:41:58.305621 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.304639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.7.105.124:33729)
0809 00:41:58.304685 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.304687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.304892 (+ 205us) server_negotiation.cc:407] Connection header received
0809 00:41:58.305012 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.305013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.305025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.305054 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.305556 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42338: BlockingRecv error: recv got EOF from 127.0.0.1:42338 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:41:58.312150 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.308921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.7.105.125:36149)
0809 00:41:58.308959 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.308961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.309226 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:41:58.309325 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.309325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.309336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.309364 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.309538 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.310237 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.311316 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.311609 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.311655 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.311865 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.311900 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.311929 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.312037 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.312076 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:41:58.315429 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.314643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.7.105.124:33729)
0809 00:41:58.314684 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.314685 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.314891 (+ 206us) server_negotiation.cc:407] Connection header received
0809 00:41:58.314997 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.314998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.315010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.315040 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.315378 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42354: BlockingRecv error: recv got EOF from 127.0.0.1:42354 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:41:58.324331 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.321093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35270 (local address 127.7.105.125:36149)
0809 00:41:58.321130 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.321132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.321390 (+ 258us) server_negotiation.cc:407] Connection header received
0809 00:41:58.321484 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.321485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.321495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.321522 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.321707 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.322410 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.323498 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.323802 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.323845 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.324045 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.324100 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.324131 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.324243 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.324261 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:41:58.327819 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.327017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.7.105.124:33729)
0809 00:41:58.327059 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.327061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.327268 (+ 207us) server_negotiation.cc:407] Connection header received
0809 00:41:58.327368 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.327369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.327381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.327411 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.327768 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42356: BlockingRecv error: recv got EOF from 127.0.0.1:42356 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:41:58.344206 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.340442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.7.105.125:36149)
0809 00:41:58.340510 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.340512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.340925 (+ 413us) server_negotiation.cc:407] Connection header received
0809 00:41:58.341047 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.341048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.341061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.341090 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.341277 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.342047 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.343237 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.343577 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.343621 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.343837 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.343882 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.343913 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.344039 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.344122 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 00:41:58.347862 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.346953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.7.105.124:33729)
0809 00:41:58.346996 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.346998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.347262 (+ 264us) server_negotiation.cc:407] Connection header received
0809 00:41:58.347362 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.347363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.347376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.347406 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.347800 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42358: BlockingRecv error: recv got EOF from 127.0.0.1:42358 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:41:58.369138 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.365569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.7.105.125:36149)
0809 00:41:58.365614 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.365616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.365892 (+ 276us) server_negotiation.cc:407] Connection header received
0809 00:41:58.365992 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.365993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.366003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.366032 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.366211 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.366977 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.368295 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.368592 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.368640 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.368825 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.368858 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.368887 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.369024 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.369044 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 00:41:58.372886 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.371972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.7.105.124:33729)
0809 00:41:58.372020 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.372022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.372288 (+ 266us) server_negotiation.cc:407] Connection header received
0809 00:41:58.372402 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.372404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.372420 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.372451 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.372829 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42370: BlockingRecv error: recv got EOF from 127.0.0.1:42370 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:41:58.408991 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.406394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35290 (local address 127.7.105.125:36149)
0809 00:41:58.406445 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.406448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.406572 (+ 124us) server_negotiation.cc:407] Connection header received
0809 00:41:58.406695 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.406696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.406707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.406775 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.406920 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.407412 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.408264 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.408461 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.408548 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.408712 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.408751 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.408810 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.408872 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.408891 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:41:58.411448 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.410648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.7.105.124:33729)
0809 00:41:58.410695 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.410697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.410897 (+ 200us) server_negotiation.cc:407] Connection header received
0809 00:41:58.411010 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.411011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.411025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.411072 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.411393 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42386: BlockingRecv error: recv got EOF from 127.0.0.1:42386 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:41:58.479435 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.476881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.7.105.125:36149)
0809 00:41:58.476936 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.476938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.477119 (+ 181us) server_negotiation.cc:407] Connection header received
0809 00:41:58.477165 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.477166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.477176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.477225 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.477365 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.477841 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.478693 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.478873 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.478923 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.479167 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.479197 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.479234 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.479309 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.479325 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 00:41:58.481865 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.481340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.7.105.124:33729)
0809 00:41:58.481391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.481392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.481440 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:41:58.481495 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.481496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.481509 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.481556 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.481820 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42392: BlockingRecv error: recv got EOF from 127.0.0.1:42392 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:41:58.613747 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.611125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.7.105.125:36149)
0809 00:41:58.611220 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.611222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.611281 (+ 59us) server_negotiation.cc:407] Connection header received
0809 00:41:58.611333 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.611334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.611344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.611398 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.611522 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.612010 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.613054 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.613238 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.613292 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.613460 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.613489 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.613527 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.613619 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.613635 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 00:41:58.616276 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.615488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.7.105.124:33729)
0809 00:41:58.615574 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.615578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.615626 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:41:58.615682 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.615683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.615694 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.615741 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.616188 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42396: BlockingRecv error: recv got EOF from 127.0.0.1:42396 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:41:58.875803 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:58.873635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35320 (local address 127.7.105.125:36149)
0809 00:41:58.873704 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.873706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.873755 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:41:58.873813 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.873813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.873822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.873873 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:58.873998 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.874488 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.875253 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:58.875433 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:58.875483 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:58.875610 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:58.875637 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:58.875674 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:58.875726 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 00:41:58.875742 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 00:41:58.878289 32613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:41:58.877732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.7.105.124:33729)
0809 00:41:58.877801 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:58.877803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:58.877811 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:58.877834 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:58.877835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:58.877845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:58.877897 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:41:58.878246 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42410: BlockingRecv error: recv got EOF from 127.0.0.1:42410 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:41:59.197968 32642 catalog_manager.cc:1259] Loaded cluster ID: 6213ff08f97940bb9821c5c6d350eeb3
I20260809 00:41:59.198056 32642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: loading cluster ID for follower catalog manager: success
I20260809 00:41:59.198954 32642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: acquiring CA information for follower catalog manager: success
I20260809 00:41:59.199294 32642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8b5eff5baa94b74895fbd61e0ca6bde: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:41:59.393814 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:59.391508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35326 (local address 127.7.105.125:36149)
0809 00:41:59.391633 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:59.391635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:59.391646 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:41:59.391698 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:59.391698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:59.391710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:59.391782 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:59.391920 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.392472 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.393241 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.393428 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.393478 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:59.393609 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:59.393638 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:59.393676 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:59.393727 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 00:41:59.393747 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260809 00:41:59.397867 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:59.395557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.7.105.124:33729)
0809 00:41:59.395703 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:59.395705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:59.395714 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:59.395743 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:59.395744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:59.395796 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:59.395864 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:59.395950 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.396474 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.397165 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.397352 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.397411 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:59.397534 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:59.397644 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:59.397702 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:59.397735 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 00:41:59.397752 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260809 00:41:59.402616 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:59.399763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.7.105.124:33729)
0809 00:41:59.399802 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:59.399804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:59.399904 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:41:59.399988 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:59.399990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:59.400002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:59.400048 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:59.400221 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.400906 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.401876 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.402169 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.402238 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:59.402385 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:59.402423 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:59.402477 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:59.402522 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 00:41:59.402543 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:41:59.403404 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:59.399573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.7.105.125:36149)
0809 00:41:59.399779 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:59.399781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:59.399789 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:41:59.399814 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:59.399814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:59.399824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:59.399904 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:59.400026 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.400801 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.402277 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.402569 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.402671 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:59.402838 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:59.402871 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:59.402923 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:59.403309 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 00:41:59.403337 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260809 00:41:59.403475 348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:41:59.399546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.7.105.126:41879)
0809 00:41:59.399805 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 00:41:59.399807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:41:59.399816 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:41:59.399866 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:41:59.399866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:41:59.399924 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:41:59.400002 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:41:59.400999 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.401473 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.402420 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:41:59.402772 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:41:59.402839 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:41:59.403084 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:41:59.403199 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:41:59.403263 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:41:59.403359 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:41:59.403382 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":57,"threads_started":1}
I20260809 00:41:59.405177 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32372
I20260809 00:41:59.414419 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32452
I20260809 00:41:59.421217 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 32534
2026-08-09T00:41:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3049 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:41:59 dist-test-slave-1zqn krb5kdc[352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:41:59 dist-test-slave-1zqn krb5kdc[352](info): set up 2 sockets
Aug 09 00:41:59 dist-test-slave-1zqn krb5kdc[352](info): commencing operation
krb5kdc: starting...
W20260809 00:42:00.582938 7589 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/test-user.keytab.
Aug 09 00:42:00 dist-test-slave-1zqn krb5kdc[352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
Entry for principal kudu/127.7.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab.
I20260809 00:42:00.711069 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40527
--webserver_interface=127.7.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:00.795497 365 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:42:00.795773 365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:00.795805 365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:00.795848 365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:00.797106 365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:00.797183 365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:00.797207 365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:00.797228 365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:00.797259 365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:00.798801 365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.126.keytab
--principal=kudu/127.7.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.126:40527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:00.799101 365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:00.799350 365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:00.804443 373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:00.804486 370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:00.804601 365 server_base.cc:1034] running on GCE node
W20260809 00:42:00.804520 371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:00.804915 365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:00.805258 365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:00.806420 365 hybrid_clock.cc:648] HybridClock initialized: now 1786236120806388 us; error 48 us; skew 500 ppm
Aug 09 00:42:00 dist-test-slave-1zqn krb5kdc[352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:00.814751 365 init.cc:377] Logged in from keytab as kudu/127.7.105.126@KRBTEST.COM (short username kudu)
I20260809 00:42:00.815256 365 webserver.cc:460] Webserver started at http://127.7.105.126:36321/ using document root <none> and password file <none>
I20260809 00:42:00.815505 365 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:00.815559 365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:00.815788 365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:00.816992 365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data/instance:
uuid: "629d09478d2a476f876bac12de2c9704"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "5fe005224284334ec2d3e43af1d4d7b5"
server_key_iv: "7de3f022917b179428f507efe04ffc78"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.817389 365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance:
uuid: "629d09478d2a476f876bac12de2c9704"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "5fe005224284334ec2d3e43af1d4d7b5"
server_key_iv: "7de3f022917b179428f507efe04ffc78"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.818898 365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:00.819768 380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:00.820045 365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:00.820207 365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal
uuid: "629d09478d2a476f876bac12de2c9704"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "5fe005224284334ec2d3e43af1d4d7b5"
server_key_iv: "7de3f022917b179428f507efe04ffc78"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.820324 365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:00.833568 365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:00.835983 365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:00.836231 365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:00.840454 365 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.126:40527
I20260809 00:42:00.840581 436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.126:40527 every 8 connection(s)
I20260809 00:42:00.840907 365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:00.841828 437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:00.843438 437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:00.846366 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 365
I20260809 00:42:00.846484 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-0/wal/instance
I20260809 00:42:00.846680 7589 external_mini_cluster.cc:1428] Setting key 75ca2f0868ae1964e8f9ce10dbfefd9f
Aug 09 00:42:00 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:42:00.853940 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.844178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40527 (local address 127.0.0.1:33210)
0809 00:42:00.844412 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.844457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:00.845017 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:00.845325 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:00.845327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:00.845638 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:00.845793 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:00.845797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:00.847025 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:00.847026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:00.847710 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:00.847712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:00.848331 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:00.848332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:00.848409 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:00.848988 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:00.848998 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:00.850386 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:00.852355 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:00.852361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:00.852363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:00.852568 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:00.852826 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:00.852827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:00.852827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:00.852932 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:00.853301 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:00.853307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:00.853369 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 00:42:00.853495 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 00:42:00.853751 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":112,"thread_start_us":108,"threads_started":1}
I20260809 00:42:00.853940 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.844270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.7.105.126:40527)
0809 00:42:00.844423 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:00.844432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:00.844578 (+ 146us) server_negotiation.cc:407] Connection header received
0809 00:42:00.845195 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:00.845197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:00.845224 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:00.845319 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:00.845914 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:00.846918 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:00.847812 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:00.848214 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:00.848345 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:00.850482 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:00.850491 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:00.850491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:00.850498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:00.852262 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:00.852664 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:00.852667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:00.852667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:00.852757 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:00.853007 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:00.853007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:00.853008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:00.853223 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:00.853311 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:00.853542 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 00:42:00.853686 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 00:42:00.854666 437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.105.125@KRBTEST.COM; defaulting to no policy
I20260809 00:42:00.855017 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.854912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41132)
0809 00:42:00.854944 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.854987 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 00:42:00.855157 385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
I20260809 00:42:00.855578 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.855481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41142)
0809 00:42:00.855532 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.855556 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:42:00.855690 437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:43303: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
Entry for principal kudu/127.7.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab.
I20260809 00:42:00.875730 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:43303
--webserver_interface=127.7.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:00.904918 437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } attempt: 1
I20260809 00:42:00.905438 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.905292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41158)
0809 00:42:00.905367 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.905408 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 00:42:00.905804 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.905704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41160)
0809 00:42:00.905752 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.905777 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:42:00.905892 437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:43303: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
W20260809 00:42:00.960275 445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:42:00.960548 445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:00.960580 445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:00.960625 445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:00.961773 445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:00.961846 445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:00.961869 445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:00.961890 445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:00.961918 445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:00.963524 445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.125.keytab
--principal=kudu/127.7.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.125:43303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:00.963835 445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:00.964125 445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:00.969538 453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:00.969532 450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:00.969617 445 server_base.cc:1034] running on GCE node
W20260809 00:42:00.969527 451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:00.970064 445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:00.970381 445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:00.971572 445 hybrid_clock.cc:648] HybridClock initialized: now 1786236120971569 us; error 40 us; skew 500 ppm
Aug 09 00:42:00 dist-test-slave-1zqn krb5kdc[352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:00.974104 437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } attempt: 2
I20260809 00:42:00.974272 445 init.cc:377] Logged in from keytab as kudu/127.7.105.125@KRBTEST.COM (short username kudu)
I20260809 00:42:00.974534 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.974396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41162)
0809 00:42:00.974457 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.974495 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:42:00.974874 445 webserver.cc:460] Webserver started at http://127.7.105.125:43455/ using document root <none> and password file <none>
I20260809 00:42:00.974956 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:00.974845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41172)
0809 00:42:00.974901 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:00.974931 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 00:42:00.975040 437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.125:43303: Network error: Client connection negotiation failed: client connection to 127.7.105.125:43303: connect: Connection refused (error 111)
I20260809 00:42:00.975179 445 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:00.975253 445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:00.975397 445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:00.976580 445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data/instance:
uuid: "188cd04079174a45b3a1e3e397455aa1"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "f554980de5616c7396f8a02ddeef1fff"
server_key_iv: "2404b87eb53641cc9573f062b5fc63f3"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.976961 445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal/instance:
uuid: "188cd04079174a45b3a1e3e397455aa1"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "f554980de5616c7396f8a02ddeef1fff"
server_key_iv: "2404b87eb53641cc9573f062b5fc63f3"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.978339 445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:00.979161 461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:00.979341 445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:00.979450 445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal
uuid: "188cd04079174a45b3a1e3e397455aa1"
format_stamp: "Formatted at 2026-08-09 00:42:00 on dist-test-slave-1zqn"
server_key: "f554980de5616c7396f8a02ddeef1fff"
server_key_iv: "2404b87eb53641cc9573f062b5fc63f3"
server_key_version: "encryptionkey@0"
I20260809 00:42:00.979538 445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:00.993844 445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:00.995814 445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:00.995994 445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:01.000252 445 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.125:43303
I20260809 00:42:01.000312 517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.125:43303 every 8 connection(s)
I20260809 00:42:01.000701 445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/data/info.pb
I20260809 00:42:01.001683 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 445
I20260809 00:42:01.001787 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-1/wal/instance
I20260809 00:42:01.002007 7589 external_mini_cluster.cc:1428] Setting key df7eb227cf4b4659bcd28a07f4c535d5
I20260809 00:42:01.002002 518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:01.003743 518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.105.124@KRBTEST.COM; defaulting to no policy
I20260809 00:42:01.014261 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.004836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.7.105.126:40527)
0809 00:42:01.004916 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.004918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.005144 (+ 226us) server_negotiation.cc:407] Connection header received
0809 00:42:01.005638 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.005639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.005658 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.005739 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.006392 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.007223 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.008278 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.008654 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.008736 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.011356 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.011358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.011359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.011368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.012871 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.013284 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.013285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.013285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.013339 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.013596 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.013597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.013598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.013723 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.013778 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.014051 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.014174 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.105.124@KRBTEST.COM" created.
I20260809 00:42:01.014390 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.004788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40527 (local address 127.0.0.1:33226)
0809 00:42:01.004985 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.005044 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.005529 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.005812 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.005816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.006086 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.006288 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.006292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.007317 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.007318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.008193 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.008196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.008768 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.008769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.008866 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.009450 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.009459 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.011237 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.012949 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.012956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.012958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.013211 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.013403 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.013403 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.013403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.013511 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.013785 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.013791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.013916 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.014003 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.014129 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260809 00:42:01.015062 518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:42:01.022544 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.015373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41184)
0809 00:42:01.015433 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.015444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.015486 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.015770 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.015771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.015916 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.015960 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.015961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.016851 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.016853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.017365 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.017366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.017853 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.017854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.017921 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.018356 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.018361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.019580 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.021470 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.021471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.021471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.021639 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.021869 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.021870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.021870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.021923 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.022218 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.022219 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.022259 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.022324 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.022426 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:42:01.022646 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.015445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.7.105.125:43303)
0809 00:42:01.015614 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.015618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.015634 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:42:01.015651 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.015653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.015663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.015766 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.016040 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.016758 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.017449 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.017763 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.017845 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.019676 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.019684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.019685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.019692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.021364 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.021733 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.021737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.021737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.021795 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.021994 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.021995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.021996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.022147 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.022242 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.022352 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.022511 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260809 00:42:01.023263 518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:01.023669 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.023540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42242)
0809 00:42:01.023592 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.023638 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:42:01.023805 463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
I20260809 00:42:01.024329 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.024239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42254)
0809 00:42:01.024274 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.024309 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 00:42:01.024458 518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:44109: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
Entry for principal kudu/127.7.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab.
I20260809 00:42:01.032663 7589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
/tmp/dist-test-taskjrm2oQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:44109
--webserver_interface=127.7.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.105.84:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:01.066193 437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } attempt: 3
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:42:01.073678 518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } attempt: 1
I20260809 00:42:01.074059 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.066623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41198)
0809 00:42:01.066691 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.066702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.066769 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.066935 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.066937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.067141 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.067213 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.067215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.067916 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.067917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.068505 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.068507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.068932 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.068934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.069017 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.069556 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.069561 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.071161 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.072881 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.072882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.072883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.073136 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.073341 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.073342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.073342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.073419 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.073682 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.073683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.073739 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.073821 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.073949 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:42:01.074097 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.073964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42264)
0809 00:42:01.074017 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.074070 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:42:01.074430 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.066707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.7.105.125:43303)
0809 00:42:01.066792 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.066794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.066817 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:42:01.066846 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.066846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.066856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.066930 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.067307 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.067818 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.068613 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.068854 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.068935 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.071258 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.071260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.071260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.071267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.072779 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.073218 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.073219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.073219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.073271 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.073486 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.073487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.073487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.073590 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.073681 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.073933 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.074022 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:42:01.074484 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.074353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42274)
0809 00:42:01.074408 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.074439 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 00:42:01.074831 518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:44109: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
I20260809 00:42:01.075114 437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:01.075501 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.075375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42286)
0809 00:42:01.075429 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.075469 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:42:01.075888 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.075768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42292)
0809 00:42:01.075827 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.075859 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 00:42:01.076004 437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:44109: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
I20260809 00:42:01.107218 437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } attempt: 1
I20260809 00:42:01.107854 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.107712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42300)
0809 00:42:01.107779 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.107825 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:42:01.108287 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.108194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42310)
0809 00:42:01.108243 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.108269 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:42:01.108436 437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:44109: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
W20260809 00:42:01.125744 526 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:42:01.125996 526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:01.126039 526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:01.126084 526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:01.127313 526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:01.127378 526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:01.127413 526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:01.127445 526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:01.127477 526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:01.129163 526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.105.84:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.105.126:40527,127.7.105.125:43303,127.7.105.124:44109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/krb5kdc/kudu_127.7.105.124.keytab
--principal=kudu/127.7.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.105.124:44109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:01.129475 526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:01.129724 526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:01.134927 532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:01.134902 531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:01.134953 535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:01.135005 526 server_base.cc:1034] running on GCE node
I20260809 00:42:01.135434 526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:01.135725 526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:01.136895 526 hybrid_clock.cc:648] HybridClock initialized: now 1786236121136899 us; error 49 us; skew 500 ppm
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:01.139700 526 init.cc:377] Logged in from keytab as kudu/127.7.105.124@KRBTEST.COM (short username kudu)
I20260809 00:42:01.140206 526 webserver.cc:460] Webserver started at http://127.7.105.124:43675/ using document root <none> and password file <none>
I20260809 00:42:01.140461 526 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:01.140530 526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:01.140746 526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:01.141829 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data/instance:
uuid: "b87ef895774d418f9fe3c5695bcd5e4c"
format_stamp: "Formatted at 2026-08-09 00:42:01 on dist-test-slave-1zqn"
server_key: "0c2dae7849fe58215a71733eb95fc79a"
server_key_iv: "47073cafac977d07245be8cab07939fa"
server_key_version: "encryptionkey@0"
I20260809 00:42:01.142200 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal/instance:
uuid: "b87ef895774d418f9fe3c5695bcd5e4c"
format_stamp: "Formatted at 2026-08-09 00:42:01 on dist-test-slave-1zqn"
server_key: "0c2dae7849fe58215a71733eb95fc79a"
server_key_iv: "47073cafac977d07245be8cab07939fa"
server_key_version: "encryptionkey@0"
I20260809 00:42:01.143046 518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } attempt: 2
I20260809 00:42:01.143460 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.143320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42322)
0809 00:42:01.143386 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.143422 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:42:01.143764 526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:01.143924 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.143796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42328)
0809 00:42:01.143862 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.143900 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:42:01.144089 518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.105.124:44109: Network error: Client connection negotiation failed: client connection to 127.7.105.124:44109: connect: Connection refused (error 111)
I20260809 00:42:01.144703 542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:01.144889 526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:01.144984 526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data,/tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal
uuid: "b87ef895774d418f9fe3c5695bcd5e4c"
format_stamp: "Formatted at 2026-08-09 00:42:01 on dist-test-slave-1zqn"
server_key: "0c2dae7849fe58215a71733eb95fc79a"
server_key_iv: "47073cafac977d07245be8cab07939fa"
server_key_version: "encryptionkey@0"
I20260809 00:42:01.145084 526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:01.153050 526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:01.155364 526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:01.155519 526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:01.159811 526 rpc_server.cc:307] RPC server started. Bound to: 127.7.105.124:44109
I20260809 00:42:01.159860 598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.105.124:44109 every 8 connection(s)
I20260809 00:42:01.160259 526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/data/info.pb
I20260809 00:42:01.161345 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:01.162981 599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:01.168545 7589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu as pid 526
I20260809 00:42:01.168692 7589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjrm2oQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235843038573-7589-0/minicluster-data/master-2/wal/instance
I20260809 00:42:01.168952 7589 external_mini_cluster.cc:1428] Setting key 2607845263d4720b705b59149375edb0
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
I20260809 00:42:01.173547 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.163701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.126:40527 (local address 127.0.0.1:33232)
0809 00:42:01.163880 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.163920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.164512 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.164788 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.164791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.165052 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.165256 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.165260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.166072 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.166073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.166965 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.166968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.167394 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.167396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.167503 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.168202 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.168211 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.169794 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.172014 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.172022 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.172025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.172297 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.172518 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.172519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.172520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.172684 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.173042 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.173048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.173177 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.173259 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.173378 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260809 00:42:01.173589 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.163766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.7.105.126:40527)
0809 00:42:01.163860 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.163863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.164042 (+ 179us) server_negotiation.cc:407] Connection header received
0809 00:42:01.164635 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.164636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.164647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.164712 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.165376 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.165953 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.167075 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.167270 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.167339 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.169896 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.169898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.169899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.169907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.171902 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.172381 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.172383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.172383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.172449 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.172789 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.172791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.172792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.172966 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.173041 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.173330 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.173429 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:42:01.174261 599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:42:01.179881 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.169401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.7.105.124:44109)
0809 00:42:01.169590 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.169596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.169655 (+ 59us) server_negotiation.cc:407] Connection header received
0809 00:42:01.169715 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.169718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.169742 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.169831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.170455 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.171132 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.171905 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.172300 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.172385 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.175430 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.175438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.175439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.175446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.178409 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.178880 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.178883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.178884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.178954 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.179259 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.179260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.179260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.179465 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.179633 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.179687 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.179767 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260809 00:42:01.181839 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.174655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.7.105.125:43303)
0809 00:42:01.174727 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.174729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.174737 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:01.174762 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.174763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.174772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.174826 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.175134 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.175603 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.176339 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.176557 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.176643 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.178893 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.178894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.178894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.178902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.180713 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.181139 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.181140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.181141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.181204 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.181382 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.181383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.181383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.181522 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.181589 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.181720 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.181779 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:42:01.181885 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.174552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.125:43303 (local address 127.0.0.1:41202)
0809 00:42:01.174608 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.174620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.174661 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.174837 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.174838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.175003 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.175058 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.175059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.175683 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.175684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.176258 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.176260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.176644 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.176645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.176718 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.177089 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.177093 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.178811 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.180887 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.180888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.180888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.181043 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.181273 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.181273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.181274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.181323 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.181590 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.181591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.181632 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.181702 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.181790 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:42:01.182245 599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:01.183645 437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } attempt: 2
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.126@KRBTEST.COM
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.124@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.126@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
I20260809 00:42:01.189568 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.180802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33238 (local address 127.7.105.126:40527)
0809 00:42:01.180841 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.180843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.180851 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:01.180874 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.180875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.180884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.180952 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.181596 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.182283 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.183335 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.183835 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.183908 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.186662 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.186664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.186664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.186672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.188328 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.188723 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.188724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.188724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.188778 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.188988 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.188989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.188989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.189109 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.189284 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.189321 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.189394 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:42:01.189817 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.182485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42348)
0809 00:42:01.182562 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.182573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.182616 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.182732 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.182733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.182837 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.182866 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.182867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.183650 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.183651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.184613 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.184615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.185045 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.185046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.185097 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.185473 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.185477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.187063 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.188804 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.188805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.188805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.189003 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.189180 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.189181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.189181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.189229 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.189499 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.189499 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.189568 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.189636 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.189725 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 00:42:01.190104 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.182555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.7.105.124:44109)
0809 00:42:01.182590 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.182592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.182600 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:01.182656 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.182657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.182665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.182730 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.182929 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.183538 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.184688 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.184891 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.185045 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.187145 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.187146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.187147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.187153 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.188725 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.189071 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.189072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.189072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.189126 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.189289 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.189290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.189291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.189418 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.189478 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.189934 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.190018 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:42:01.191766 599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: Bootstrap starting.
I20260809 00:42:01.192353 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.183919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42352)
0809 00:42:01.183956 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.183965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.184004 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.184301 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.184303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.184454 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.184490 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.184491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.185728 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.185729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.186302 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.186304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.186749 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.186750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.186820 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.187382 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.187387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.188920 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.191261 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.191262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.191263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.191433 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.191639 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.191640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.191641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.191705 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.192013 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.192013 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.192079 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.192153 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.192249 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:42:01.192384 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.183944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.7.105.124:44109)
0809 00:42:01.184149 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.184151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.184157 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:42:01.184182 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.184182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.184189 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.184277 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.184559 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.185646 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.186383 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.186604 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.186722 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.189008 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.189009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.189009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.189015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.191157 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.191506 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.191507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.191507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.191564 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.191780 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.191781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.191781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.191933 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.192018 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.192180 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.192251 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":138,"threads_started":1}
I20260809 00:42:01.192581 599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:01.192878 599 log.cc:824] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:01.193847 437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: Bootstrap starting.
I20260809 00:42:01.194376 437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:01.194650 437 log.cc:824] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.105.125@KRBTEST.COM
I20260809 00:42:01.195502 437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: No bootstrap required, opened a new log
I20260809 00:42:01.195513 599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: No bootstrap required, opened a new log
I20260809 00:42:01.197436 599 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.197570 599 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87ef895774d418f9fe3c5695bcd5e4c, State: Initialized, Role: FOLLOWER
I20260809 00:42:01.197631 599 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.197892 609 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.197860 437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.197970 599 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:01.197984 609 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.198004 437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629d09478d2a476f876bac12de2c9704, State: Initialized, Role: FOLLOWER
I20260809 00:42:01.198109 437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.198488 437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:01.198427 610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.198577 610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.199483 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.190251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.7.105.125:43303)
0809 00:42:01.190426 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.190428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.190510 (+ 82us) server_negotiation.cc:407] Connection header received
0809 00:42:01.190595 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.190596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.190608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.190660 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.191225 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.191871 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.192849 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.193155 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.193221 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.195801 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.195802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.195803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.195811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.197854 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.198416 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.198417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.198418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.198482 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.198868 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.198869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.198870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.199027 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.199240 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.199288 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.199386 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 00:42:01.201037 7589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 00:42:01.201009 635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:01.201078 635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:01.201902 409 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "629d09478d2a476f876bac12de2c9704"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33238
I20260809 00:42:01.202034 409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:42:01.202101 409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:01.203142 409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.200371 526 master_runner.cc:186] Error getting master registration for 127.7.105.126:40527: OK, instance_id { permanent_uuid: "629d09478d2a476f876bac12de2c9704" instance_seqno: 1786236120839197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:42:01.203290 526 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:42:01.203405 409 leader_election.cc:288] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [CANDIDATE]: Term 1 election: Requested vote from peers 188cd04079174a45b3a1e3e397455aa1 (127.7.105.125:43303), b87ef895774d418f9fe3c5695bcd5e4c (127.7.105.124:44109)
I20260809 00:42:01.203712 571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629d09478d2a476f876bac12de2c9704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b87ef895774d418f9fe3c5695bcd5e4c"
W20260809 00:42:01.203792 628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:01.203835 628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:01.203836 571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:01.205076 571 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629d09478d2a476f876bac12de2c9704 in term 1.
I20260809 00:42:01.205322 382 leader_election.cc:302] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629d09478d2a476f876bac12de2c9704, b87ef895774d418f9fe3c5695bcd5e4c; no voters:
I20260809 00:42:01.205442 610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:01.205587 610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [term 1 LEADER]: Becoming Leader. State: Replica: 629d09478d2a476f876bac12de2c9704, State: Running, Role: LEADER
I20260809 00:42:01.205682 610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.206135 610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629d09478d2a476f876bac12de2c9704. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.206246 610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:01.204547 365 master_runner.cc:186] Error getting master registration for 127.7.105.125:43303: OK, instance_id { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" instance_seqno: 1786236120998976 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:42:01.207096 365 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:42:01.209800 640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:01.210026 640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:01.217300 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.208171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.7.105.126:40527)
0809 00:42:01.208211 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.208213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.208517 (+ 304us) server_negotiation.cc:407] Connection header received
0809 00:42:01.208636 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.208637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.208648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.208711 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.209551 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.210375 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.211514 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.211823 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.211893 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.213506 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.213508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.213508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.213516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.215356 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.216235 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.216236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.216237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.216299 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.216533 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.216534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.216535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.216677 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.217084 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.217128 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.217207 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:01.223261 571 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [term 1 FOLLOWER]: Refusing update from remote peer 629d09478d2a476f876bac12de2c9704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:01.223632 638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:01.224815 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.208274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.7.105.125:43303)
0809 00:42:01.208397 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.208399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.212314 (+ 3915us) server_negotiation.cc:407] Connection header received
0809 00:42:01.212436 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.212437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.212448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.212478 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.214060 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.214782 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.215859 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.216211 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.216279 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.219386 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.219387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.219388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.219395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.221477 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.222696 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.222698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.222698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.222761 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.224008 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.224009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.224009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.224206 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.224303 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.224629 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.224721 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 00:42:01.225312 609 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629d09478d2a476f876bac12de2c9704. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.225412 609 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.226349 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.208782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.7.105.124:44109)
0809 00:42:01.208838 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.208840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.217159 (+ 8319us) server_negotiation.cc:407] Connection header received
0809 00:42:01.217262 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.217263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.217275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.217308 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.218423 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.219211 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.220280 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.220616 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.220658 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.222370 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.222372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.222372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.222380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.224319 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.225241 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.225243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.225243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.225303 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.225696 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.225697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.225697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.225828 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.225918 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.226173 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.226260 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:42:01.233731 610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.233829 610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:01.233943 609 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.234030 609 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.234221 649 mvcc.cc:205] Tried to move back new op lower bound from 7316423152522141696 to 7316423152459210752. Current Snapshot: MvccSnapshot[applied={T|T < 7316423152522141696}]
I20260809 00:42:01.234323 650 mvcc.cc:205] Tried to move back new op lower bound from 7316423152522141696 to 7316423152459210752. Current Snapshot: MvccSnapshot[applied={T|T < 7316423152522141696}]
I20260809 00:42:01.235204 640 catalog_manager.cc:1347] Generated new cluster ID: cfdb71ed7abc4819b6e40d8d0fce6ff0
I20260809 00:42:01.235255 640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:01.240085 518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } attempt: 3
Aug 09 00:42:01 dist-test-slave-1zqn krb5kdc[352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.105.125@KRBTEST.COM for kudu/127.7.105.124@KRBTEST.COM
I20260809 00:42:01.250804 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.240363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.105.124:44109 (local address 127.0.0.1:42368)
0809 00:42:01.240451 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:01.240463 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:01.240500 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:01.240733 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:01.240733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:01.240916 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:01.240971 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.240973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.242102 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.242103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.242935 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:01.242938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.243444 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.243445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:01.243503 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.244013 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:01.244018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:01.246436 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:01.249338 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.249339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.249339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.249582 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.249917 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:01.249918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:01.249918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:01.249978 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:01.250469 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:01.250470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:01.250527 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:42:01.250583 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:42:01.250683 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 00:42:01.251451 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.240578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.7.105.124:44109)
0809 00:42:01.240615 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.240617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.240623 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:42:01.240643 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.240644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.240653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.241039 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.241074 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.242000 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.243062 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.243356 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.243696 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.246526 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:01.246527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:01.246528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:01.246535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:01.249196 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.249713 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.249714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.249715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.249804 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:01.250065 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:01.250066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:01.250066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:01.250328 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:01.251036 (+ 708us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.251111 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.251298 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:01.253089 518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: Bootstrap starting.
I20260809 00:42:01.253872 518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:01.254195 518 log.cc:824] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:01.255613 640 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:01.256394 640 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:01.260771 518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: No bootstrap required, opened a new log
I20260809 00:42:01.262491 640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704: Generated new TSK 0
I20260809 00:42:01.262715 640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:01.262992 518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.263149 518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188cd04079174a45b3a1e3e397455aa1, State: Initialized, Role: FOLLOWER
I20260809 00:42:01.263232 518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } }
I20260809 00:42:01.263525 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.263605 518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:01.263614 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.263697 491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "629d09478d2a476f876bac12de2c9704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "188cd04079174a45b3a1e3e397455aa1"
I20260809 00:42:01.263818 491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:01.264966 491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629d09478d2a476f876bac12de2c9704 in term 1.
I20260809 00:42:01.265566 490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [term 1 FOLLOWER]: Refusing update from remote peer 629d09478d2a476f876bac12de2c9704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:01.266139 638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:01.268297 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629d09478d2a476f876bac12de2c9704. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.268390 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.268425 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.268463 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:01.272732 668 mvcc.cc:205] Tried to move back new op lower bound from 7316423152685977600 to 7316423152459210752. Current Snapshot: MvccSnapshot[applied={T|T < 7316423152522141696}]
I20260809 00:42:01.276780 667 catalog_manager.cc:1259] Loaded cluster ID: cfdb71ed7abc4819b6e40d8d0fce6ff0
I20260809 00:42:01.276840 667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: loading cluster ID for follower catalog manager: success
I20260809 00:42:01.278018 667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: acquiring CA information for follower catalog manager: success
I20260809 00:42:01.278564 667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 188cd04079174a45b3a1e3e397455aa1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:42:01.290892 523 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.289621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.7.105.125:43303)
0809 00:42:01.289667 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.289669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.290264 (+ 595us) server_negotiation.cc:407] Connection header received
0809 00:42:01.290363 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.290364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.290375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.290405 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.290828 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41232: BlockingRecv error: recv got EOF from 127.0.0.1:41232 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:42:01.291765 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.290024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.7.105.124:44109)
0809 00:42:01.290081 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.290083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.291283 (+ 1200us) server_negotiation.cc:407] Connection header received
0809 00:42:01.291375 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.291376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.291387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.291414 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.291704 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42370: BlockingRecv error: recv got EOF from 127.0.0.1:42370 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 00:42:01.292920 440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.292340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.7.105.126:40527)
0809 00:42:01.292390 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.292392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.292427 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:42:01.292518 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.292519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.292530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.292559 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.292858 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33262: BlockingRecv error: recv got EOF from 127.0.0.1:33262 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:01.300462 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.296726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.7.105.125:43303)
0809 00:42:01.296781 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.296783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.296995 (+ 212us) server_negotiation.cc:407] Connection header received
0809 00:42:01.297122 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.297123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.297170 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.297214 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.297432 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.298162 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.299357 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.299635 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.299683 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.299994 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.300172 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.300231 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.300360 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.300382 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:42:01.304127 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.303246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.7.105.124:44109)
0809 00:42:01.303288 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.303290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.303506 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:42:01.303623 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.303624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.303644 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.303676 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.304077 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42384: BlockingRecv error: recv got EOF from 127.0.0.1:42384 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:01.309689 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.306507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.7.105.125:43303)
0809 00:42:01.306543 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.306545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.306783 (+ 238us) server_negotiation.cc:407] Connection header received
0809 00:42:01.306872 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.306873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.306882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.306908 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.307075 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.307769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.308917 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.309193 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.309236 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.309436 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.309470 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.309497 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.309603 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.309622 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:42:01.313199 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.312414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.7.105.124:44109)
0809 00:42:01.312454 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.312456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.312662 (+ 206us) server_negotiation.cc:407] Connection header received
0809 00:42:01.312759 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.312760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.312771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.312799 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.313151 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42392: BlockingRecv error: recv got EOF from 127.0.0.1:42392 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:01.319708 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.316528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.7.105.125:43303)
0809 00:42:01.316575 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.316578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.316791 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:42:01.316888 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.316889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.316899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.316927 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.317121 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.317798 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.318866 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.319156 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.319205 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.319419 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.319456 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.319486 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.319615 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.319636 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:42:01.323374 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.322403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.7.105.124:44109)
0809 00:42:01.322443 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.322461 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.322737 (+ 276us) server_negotiation.cc:407] Connection header received
0809 00:42:01.322836 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.322838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.322849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.322879 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.323319 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42398: BlockingRecv error: recv got EOF from 127.0.0.1:42398 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:01.333199 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.329886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.7.105.125:43303)
0809 00:42:01.329927 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.329929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.330203 (+ 274us) server_negotiation.cc:407] Connection header received
0809 00:42:01.330302 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.330303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.330313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.330342 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.330507 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.331218 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.332379 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.332654 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.332701 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.332911 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.332946 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.332975 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.333101 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.333121 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 00:42:01.336566 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.335703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.7.105.124:44109)
0809 00:42:01.335746 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.335748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.335982 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:42:01.336123 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.336124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.336136 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.336166 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.336514 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42400: BlockingRecv error: recv got EOF from 127.0.0.1:42400 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:01.353083 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.345985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.7.105.125:43303)
0809 00:42:01.346044 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.346046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.349834 (+ 3788us) server_negotiation.cc:407] Connection header received
0809 00:42:01.349945 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.349946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.349959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.349989 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.350208 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.350999 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.352228 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.352520 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.352561 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.352795 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.352832 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.352858 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.352961 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.352982 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 00:42:01.357331 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.356453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.7.105.124:44109)
0809 00:42:01.356496 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.356498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.356744 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:42:01.356843 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.356844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.356856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.356886 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.357272 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42414: BlockingRecv error: recv got EOF from 127.0.0.1:42414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:01.378270 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.374818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.7.105.125:43303)
0809 00:42:01.374866 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.374868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.375173 (+ 305us) server_negotiation.cc:407] Connection header received
0809 00:42:01.375276 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.375277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.375289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.375317 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.375497 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.376302 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.377440 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.377733 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.377772 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.377982 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.378032 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.378059 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.378161 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.378182 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:42:01.381985 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.381123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.7.105.124:44109)
0809 00:42:01.381169 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.381171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.381407 (+ 236us) server_negotiation.cc:407] Connection header received
0809 00:42:01.381508 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.381509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.381522 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.381552 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.381905 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42422: BlockingRecv error: recv got EOF from 127.0.0.1:42422 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:01.418793 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.415691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.7.105.125:43303)
0809 00:42:01.415751 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.415753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.415862 (+ 109us) server_negotiation.cc:407] Connection header received
0809 00:42:01.415928 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.415929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.415941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.416002 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.416201 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.416969 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.417794 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.418175 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.418270 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.418446 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.418483 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.418541 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.418680 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.418702 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 00:42:01.421620 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.420989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.7.105.124:44109)
0809 00:42:01.421043 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.421045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.421094 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:42:01.421177 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.421178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.421192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.421254 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.421555 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42424: BlockingRecv error: recv got EOF from 127.0.0.1:42424 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:42:01.489624 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.487056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.7.105.125:43303)
0809 00:42:01.487158 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.487160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.487274 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:42:01.487401 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.487403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.487414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.487491 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.487694 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.488212 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.489044 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.489236 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.489287 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.489418 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.489444 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.489482 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.489544 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.489560 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 00:42:01.492281 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.491501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.7.105.124:44109)
0809 00:42:01.491565 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.491566 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.491597 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:42:01.491678 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.491679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.491688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.491737 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.492206 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42426: BlockingRecv error: recv got EOF from 127.0.0.1:42426 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:42:01.624120 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.621486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.7.105.125:43303)
0809 00:42:01.621571 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.621573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.621664 (+ 91us) server_negotiation.cc:407] Connection header received
0809 00:42:01.621771 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.621772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.621784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.621861 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.622007 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.622528 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.623334 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.623536 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.623619 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.623774 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.623819 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.623882 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.623947 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.623966 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 00:42:01.626863 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.626109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.7.105.124:44109)
0809 00:42:01.626189 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.626191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.626308 (+ 117us) server_negotiation.cc:407] Connection header received
0809 00:42:01.626387 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.626388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.626402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.626453 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.626792 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42434: BlockingRecv error: recv got EOF from 127.0.0.1:42434 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:42:01.687235 610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "629d09478d2a476f876bac12de2c9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629d09478d2a476f876bac12de2c9704" member_type: VOTER last_known_addr { host: "127.7.105.126" port: 40527 } } peers { permanent_uuid: "188cd04079174a45b3a1e3e397455aa1" member_type: VOTER last_known_addr { host: "127.7.105.125" port: 43303 } } peers { permanent_uuid: "b87ef895774d418f9fe3c5695bcd5e4c" member_type: VOTER last_known_addr { host: "127.7.105.124" port: 44109 } } }
I20260809 00:42:01.687434 610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629d09478d2a476f876bac12de2c9704 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:01.886729 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:01.884112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.7.105.125:43303)
0809 00:42:01.884193 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.884196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.884283 (+ 87us) server_negotiation.cc:407] Connection header received
0809 00:42:01.884381 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.884382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.884405 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.884478 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:01.884645 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.885167 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.885994 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:01.886196 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:01.886276 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:01.886454 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:01.886492 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:01.886554 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:01.886611 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 00:42:01.886632 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 00:42:01.889353 605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:01.888707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.7.105.124:44109)
0809 00:42:01.888794 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:01.888795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:01.888806 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:01.888851 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:01.888852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:01.888862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:01.888914 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:01.889298 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42438: BlockingRecv error: recv got EOF from 127.0.0.1:42438 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:42:02.204453 628 catalog_manager.cc:1259] Loaded cluster ID: cfdb71ed7abc4819b6e40d8d0fce6ff0
I20260809 00:42:02.204535 628 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: loading cluster ID for follower catalog manager: success
I20260809 00:42:02.205372 628 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: acquiring CA information for follower catalog manager: success
I20260809 00:42:02.205649 628 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b87ef895774d418f9fe3c5695bcd5e4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:42:02.404942 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:02.402560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.7.105.125:43303)
0809 00:42:02.402674 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:02.402676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:02.402769 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:42:02.402821 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:02.402822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:02.402838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:02.402901 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:02.403020 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.403543 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.404301 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.404501 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.404570 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:02.404686 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:02.404715 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:02.404754 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:02.404800 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:42:02.404818 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":52,"threads_started":1}
I20260809 00:42:02.409902 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:02.406813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.7.105.124:44109)
0809 00:42:02.406978 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:02.406981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:02.406992 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:02.407019 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:02.407020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:02.407101 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:02.407219 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:02.407341 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.407899 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.408925 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.409153 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.409258 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:02.409476 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:02.409600 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:02.409704 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:02.409762 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:42:02.409788 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260809 00:42:02.414808 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:02.411802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.7.105.125:43303)
0809 00:42:02.411845 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:02.411847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:02.412192 (+ 345us) server_negotiation.cc:407] Connection header received
0809 00:42:02.412303 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:02.412304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:02.412316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:02.412344 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:02.412508 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.413231 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.413982 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.414276 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.414336 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:02.414512 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:02.414545 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:02.414573 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:02.414688 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:42:02.414715 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:02.414928 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:02.411776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.7.105.126:40527)
0809 00:42:02.411902 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:02.411904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:02.412003 (+ 99us) server_negotiation.cc:407] Connection header received
0809 00:42:02.412112 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:02.412113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:02.412171 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:02.412286 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:02.412382 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.412926 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.413994 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.414265 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.414340 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:02.414548 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:02.414659 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:02.414752 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:02.414800 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:42:02.414827 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260809 00:42:02.415228 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:02.411858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.7.105.124:44109)
0809 00:42:02.411924 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:02.411926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:02.412174 (+ 248us) server_negotiation.cc:407] Connection header received
0809 00:42:02.412248 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:02.412249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:02.412258 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:02.412305 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:02.412441 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.413162 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.414119 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:02.414418 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:02.414480 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:02.414713 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:02.414745 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:02.414800 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:02.415135 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 00:42:02.415161 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:42:02.416743 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 365
I20260809 00:42:02.425348 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 445
I20260809 00:42:02.432714 7589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjrm2oQ/build/release/bin/kudu with pid 526
2026-08-09T00:42:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3012 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6062 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (279404 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST